Sequential recommendation

On this page 7

Sequential recommendation is the task of predicting the next item a user will interact with from the ordered history of that user’s previous interactions, such as clicks, views, plays or purchases. In recommender systems, it treats behavior as a time-ordered sequence rather than an unordered set, so that recent actions, transitions between items and evolving interests shape what is shown next.

Classic collaborative filtering and matrix factorization capture long-term preferences but ignore order: a user who just bought a camera receives the same list as before the purchase. Sequential models address this gap and now power many feeds, playlists and product carousels where short-term intent changes from one session to the next.

How sequential recommendation models user behavior

A sequential recommender encodes the history of a user as a sequence of item embeddings, often enriched with timestamps, action types or context, and produces a user state vector that scores every candidate item. Training typically uses next-item prediction: given the first t interactions, the model learns to rank interaction t+1 above sampled or all other items.

The model must balance two signals. Short-term dynamics describe what the user is doing now, for example browsing hiking boots. Long-term preferences describe stable tastes, such as a preference for a certain brand or price range. Strong systems combine both, sometimes with a separate long-term profile embedding.

The following table lists the main inputs a sequential model can use and what each contributes.

InputExampleWhat it contributes
Item identifiersProduct or track IDsCore transition patterns
Positions or timestampsOrder index, time gapsRecency and pacing of interest
Action typeView, add to cart, purchaseStrength of intent
Item attributesCategory, brand, priceGeneralization to rare items
ContextDevice, time of daySituational relevance
Typical inputs to a sequential recommender

Main model families for sequential recommendation

Sequential recommendation methods evolved from Markov chains to neural sequence models and transformers. Early approaches such as FPMC combined matrix factorization with first-order transitions, assuming that the next item depends mainly on the last one. This works well for short, sparse histories but misses longer patterns.

Neural methods learn richer dependencies. GRU4Rec applied recurrent networks to sessions, Caser used convolutional filters over recent items, and SASRec introduced causal self-attention that attends to all previous items with learned weights. BERT4Rec trains a bidirectional transformer with a masked-item objective, and newer generative recommendation models predict semantic item identifiers token by token.

FamilyRepresentative modelsStrengthWeakness
Markov chain modelsFPMC, FossilSimple, data-efficientShort memory
Recurrent networksGRU4RecNatural for sessionsSlow training, weaker on long sequences
Convolutional modelsCaser, NextItNetCaptures local patternsFixed receptive field
Self-attention modelsSASRec, BERT4RecStrong accuracy, parallel trainingQuadratic cost in sequence length
Generative and LLM-basedSemantic ID decodersUses item content and languageHigh serving cost, still maturing
Comparison of sequential recommendation model families

Sequential recommendation overlaps with several neighboring tasks, and the distinction lies mainly in what the history contains and what is predicted. Session-based recommendation is often treated as a special case in which the user is anonymous and only the current session is known.

TaskHistory usedPrediction target
Sequential recommendationFull ordered user historyNext item
Session-based recommendationCurrent anonymous sessionNext item in the session
Next-basket recommendationSequence of past basketsSet of items in the next basket
Top-N recommendationUnordered interaction setRanked list of relevant items
CTR predictionFeatures, often including sequencesClick probability of one candidate
How sequential recommendation relates to neighboring tasks

In production, sequential models frequently serve as the candidate generation stage of a multi-stage ranking pipeline, while a separate ranking model scores the shortlist with richer features.

Training, evaluation and design choices

Evaluation usually follows a leave-one-out protocol: the last interaction of each user is held out for testing and the second-to-last for validation. Metrics such as Hit Rate at K and NDCG at K measure whether the held-out item appears near the top of the ranked list.

Two practices strongly affect reported results. Sampled metrics, which rank the true item against only 100 random negatives, can reorder models compared with full ranking over the catalog, so full-catalog evaluation is preferred. The training loss matters as well: SASRec trained with full cross-entropy loss often matches or beats BERT4Rec, while binary loss with one negative underperforms.

ChoiceCommon settingEffect
Maximum sequence length50-200 itemsLonger captures more history but raises cost
Loss functionCross-entropy or sampled softmaxStrongly influences accuracy
Negative samplingUniform or popularity-basedAffects popularity bias
Evaluation splitLeave-one-out or time-basedTime-based avoids future leakage
Key design choices and their effects

Limitations and common pitfalls

Sequential recommenders inherit the general problems of recommender systems and add a few of their own. The most serious risk is data leakage: random or leave-one-out splits can let a model learn from interactions that happened after the test period.

  • Cold start for new items, because pure ID-based models cannot embed items they have never seen.
  • Repetition bias, where the model keeps recommending items similar to the last one and narrows exposure toward a filter bubble.
  • Popularity bias, since frequent items dominate transitions and training targets.
  • Serving latency, as long sequences and large catalogs make real-time scoring expensive.
  • Overfitting to short sessions, which ignores stable long-term preferences.

Put simply

Most people do things in a sensible order. After buying a tent, a person may look for sleeping bags, then a camping stove. Sequential recommendation pays attention to that order and asks a simple question: given what this person did just now, what will they most likely want next?

Older systems looked at everything a person ever liked as one big pile. A sequential system reads the history more like a story with a beginning and a recent chapter, so its suggestions follow the moment instead of only the overall taste.

In practice

A music streaming service wants its autoplay feature to choose the next track after a listener’s chosen song ends. Its earlier matrix factorization model recommended tracks matching each listener’s overall taste, so a user who switched from rock to calm piano at night still received rock songs.

The team trains a SASRec-style transformer on the last 100 plays of each listener, adding time-of-day and skip signals as context. It evaluates the model on the final week of data held out by time, ranking the true next track against the full catalog of several million tracks.

In an online test, the new model reduces early skips within the first 30 seconds and increases listening time per session. The team keeps the older model for the home page, where long-term taste matters more than the current mood.