Generative recommendation

Also known as: generative retrieval for recommendation

On this page 7

Generative recommendation is an approach to recommender systems in which a sequence model generates the identifiers of the next items a user is likely to want, token by token, instead of retrieving and scoring candidates from a fixed index. Also called generative retrieval for recommendation, it treats the catalog as a vocabulary and recommendation as conditional sequence generation, much as a language model produces the next word.

In artificial intelligence practice, the idea matters because it can merge candidate generation and ranking into one learned model, scale with model size in a way similar to large language models, and share knowledge across items through structured item codes. It also introduces new problems, such as invalid identifiers, costly decoding and difficult catalog updates.

How generative recommendation produces items

A generative recommender encodes the user context and then decodes item identifiers autoregressively. The context is usually the ordered history of interactions, such as clicks, views, purchases or ratings, sometimes enriched with text, time and device features. The decoder emits one token at a time, and each completed token sequence names exactly one catalog item.

The key design decision is how items become tokens. Assigning one atomic token per item creates a huge output vocabulary and gives rare items almost no training signal. Most modern systems therefore use semantic IDs: short tuples of discrete codes, often 3 to 4 levels, learned so that similar items share code prefixes. A residual quantizer compresses each item’s content embedding into such a tuple, and the model learns to generate the tuple.

At inference time, the system runs beam search over the code tree and keeps the top sequences as the recommendation list. Constrained decoding with a prefix trie restricts every step to codes that exist in the current catalog, which prevents the model from producing identifiers that map to no item.

The table below summarizes the main components of a typical generative recommendation pipeline and what each one controls.

ComponentRoleTypical choice
Item tokenizerMaps each item to a code sequenceResidual quantization of content embeddings
Context encoderRepresents the user historyTransformer over interaction tokens
DecoderGenerates item codes step by stepAutoregressive Transformer decoder
Decoding strategyProduces a ranked list of itemsBeam search with 10 to 100 beams
Validity constraintBlocks nonexistent identifiersPrefix trie over valid semantic IDs
Components of a generative recommendation pipeline

Item identifiers and model variants

Generative recommendation is not one architecture but a family, and the identifier scheme is the clearest way to tell its members apart. The scheme determines vocabulary size, how well knowledge transfers between items and how easily new items can be added.

Research systems such as TIGER popularized semantic IDs built with an RQ-VAE, while large industrial models, such as Meta’s HSTU generative recommender, formulate ranking and retrieval as sequential transduction over very long interaction streams. Language-model-based variants instead use textual identifiers, such as titles, and connect to the neighboring Term LLM-based recommendation.

Identifier schemeStrengthWeakness
Atomic item IDsSimple and exactHuge vocabulary and weak cold start
Semantic IDsShared prefixes transfer knowledgeQuantizer must be retrained as content shifts
Textual identifiersReuse pretrained language knowledgeAmbiguous titles and long sequences
Hierarchical cluster IDsNatural coarse-to-fine decodingClusters can become unbalanced
Collaborative IDsEncode co-interaction structureNeed sufficient interaction history
Item identifier schemes compared

Hybrid schemes are common in practice: a semantic prefix plus a disambiguating suffix guarantees that two items with identical content still receive distinct identifiers.

How it differs from retrieve-and-rank systems

Generative recommendation replaces nearest-neighbor search over an embedding index with direct generation. In a classic multi-stage ranking stack, a two-tower model embeds users and items separately, an approximate nearest-neighbor index returns hundreds of candidates, and a heavier ranker such as DeepFM or a deep learning recommendation model reorders them.

The generative model, in contrast, conditions every decoding step on the full user context, so the item representation interacts with the user representation far more deeply than a single dot product allows. It also removes the separate index, although many production systems still keep a ranker after the generator.

ApproachHow items are foundMain limitation
Generative recommendationDecodes item codes from the user contextHigh decoding cost and ID maintenance
Two-tower retrievalDot product plus nearest-neighbor indexShallow user-item interaction
Sequential recommendationScores all items from a sequence encoderSoftmax cost grows with catalog size
Matrix factorizationLatent factors of the user-item matrixIgnores order and content
LLM-based recommendationPrompts a language model with historyHallucinated items and latency
Generative recommendation versus related approaches

Sequential models such as SASRec and BERT4Rec are close relatives: they also read ordered histories, but they score items discriminatively rather than generating identifiers, so they are often used as the baseline that generative systems must beat.

Training and serving in practice

Training follows next-item prediction with teacher forcing: the model sees a user history and learns to output the code sequence of the item that actually came next. Cross-entropy loss over code tokens replaces the sampled softmax and negative sampling used in many retrieval models, although auxiliary contrastive losses are sometimes added.

Serving is where the main engineering cost appears. Beam search multiplies computation by the beam width, and latency budgets in feeds or e-commerce pages are often tens of milliseconds. Teams use key-value caching, shorter code lengths, smaller decoders and batching to fit the budget.

The following decision table shows when generative recommendation is a reasonable choice and when an established stack is the safer option.

SituationRecommended choiceReason
Large catalog with rich item contentGenerative model with semantic IDsShared prefixes help long-tail items
Very strict latency budgetTwo-tower retrieval plus rankerIndex lookup is cheaper than beam search
Rapidly changing catalogHybrid with retrieval fallbackNew items need codes before generation
Long, rich interaction historiesGenerative sequential modelDeep context modeling pays off
Small catalog and little dataMatrix factorization or SASRecGenerative gains rarely justify the cost
When to choose generative recommendation

Limitations and common mistakes

Generative recommendation is promising but not a universal replacement for retrieval. Reported gains depend heavily on datasets, tokenizer quality and how fairly baselines were tuned, and offline improvements do not always survive online A/B tests.

  • Unconstrained decoding produces identifiers that match no item, so a validity trie or post-filter is required.
  • Stale semantic IDs appear when item content changes but codes are not refreshed, which silently degrades relevance.
  • Popularity bias can be amplified because frequent code prefixes dominate beam search.
  • Low diversity results when beams converge on one prefix; diverse beam search or re-ranking helps.
  • Weak baselines in experiments exaggerate gains; compare against a well-tuned SASRec or two-tower system.

A further risk is the recommendation feedback loop: a model that generates from its own past exposures can narrow what users see over time, so exploration and beyond-accuracy metrics such as catalog coverage remain essential.

The simple explanation

Most recommendation engines work like a librarian who pulls a few hundred books off the shelves and then sorts them. Generative recommendation works differently. Its librarian has memorized a short code for every book. It simply writes down the codes of the books you will probably enjoy next.

Similar books get similar codes. That lets the system guess well even for rare items. What it learned about one book carries over to its neighbors. The catch is cost. Writing out codes piece by piece takes computing power, and the code list must stay current as the catalog changes.

How it is used in practice

A video streaming service with about two million titles wants better recommendations for niche documentaries. Its existing stack uses a two-tower model to fetch 500 candidates and a ranker to order them, but long-tail titles rarely reach the candidate set.

The team builds a generative recommender: each title receives a four-level semantic ID derived from its description, genre tags and trailer embedding, and a Transformer learns to generate the next title from each viewer’s last 200 interactions. Beam search with 50 beams and a validity trie produces candidates that are merged with the two-tower results before ranking.

In a four-week online test, the hybrid system raised viewing of long-tail titles and modestly improved completed views, while latency stayed within budget thanks to caching. The team keeps the old retrieval path as a fallback for newly added titles until their codes are assigned.