Generative recommendation is an approach to recommender systems in which a sequence model generates the identifiers of the next items a user is likely to want, token by token, instead of retrieving and scoring candidates from a fixed index. Also called generative retrieval for recommendation, it treats the catalog as a vocabulary and recommendation as conditional sequence generation, much as a language model produces the next word.
In artificial intelligence practice, the idea matters because it can merge candidate generation and ranking into one learned model, scale with model size in a way similar to large language models, and share knowledge across items through structured item codes. It also introduces new problems, such as invalid identifiers, costly decoding and difficult catalog updates.
How generative recommendation produces items
A generative recommender encodes the user context and then decodes item identifiers autoregressively. The context is usually the ordered history of interactions, such as clicks, views, purchases or ratings, sometimes enriched with text, time and device features. The decoder emits one token at a time, and each completed token sequence names exactly one catalog item.
The key design decision is how items become tokens. Assigning one atomic token per item creates a huge output vocabulary and gives rare items almost no training signal. Most modern systems therefore use semantic IDs: short tuples of discrete codes, often 3 to 4 levels, learned so that similar items share code prefixes. A residual quantizer compresses each item’s content embedding into such a tuple, and the model learns to generate the tuple.
At inference time, the system runs beam search over the code tree and keeps the top sequences as the recommendation list. Constrained decoding with a prefix trie restricts every step to codes that exist in the current catalog, which prevents the model from producing identifiers that map to no item.
The table below summarizes the main components of a typical generative recommendation pipeline and what each one controls.
| Component | Role | Typical choice |
|---|---|---|
| Item tokenizer | Maps each item to a code sequence | Residual quantization of content embeddings |
| Context encoder | Represents the user history | Transformer over interaction tokens |
| Decoder | Generates item codes step by step | Autoregressive Transformer decoder |
| Decoding strategy | Produces a ranked list of items | Beam search with 10 to 100 beams |
| Validity constraint | Blocks nonexistent identifiers | Prefix trie over valid semantic IDs |
Item identifiers and model variants
Generative recommendation is not one architecture but a family, and the identifier scheme is the clearest way to tell its members apart. The scheme determines vocabulary size, how well knowledge transfers between items and how easily new items can be added.
Research systems such as TIGER popularized semantic IDs built with an RQ-VAE, while large industrial models, such as Meta’s HSTU generative recommender, formulate ranking and retrieval as sequential transduction over very long interaction streams. Language-model-based variants instead use textual identifiers, such as titles, and connect to the neighboring Term LLM-based recommendation.
| Identifier scheme | Strength | Weakness |
|---|---|---|
| Atomic item IDs | Simple and exact | Huge vocabulary and weak cold start |
| Semantic IDs | Shared prefixes transfer knowledge | Quantizer must be retrained as content shifts |
| Textual identifiers | Reuse pretrained language knowledge | Ambiguous titles and long sequences |
| Hierarchical cluster IDs | Natural coarse-to-fine decoding | Clusters can become unbalanced |
| Collaborative IDs | Encode co-interaction structure | Need sufficient interaction history |
Hybrid schemes are common in practice: a semantic prefix plus a disambiguating suffix guarantees that two items with identical content still receive distinct identifiers.
How it differs from retrieve-and-rank systems
Generative recommendation replaces nearest-neighbor search over an embedding index with direct generation. In a classic multi-stage ranking stack, a two-tower model embeds users and items separately, an approximate nearest-neighbor index returns hundreds of candidates, and a heavier ranker such as DeepFM or a deep learning recommendation model reorders them.
The generative model, in contrast, conditions every decoding step on the full user context, so the item representation interacts with the user representation far more deeply than a single dot product allows. It also removes the separate index, although many production systems still keep a ranker after the generator.
| Approach | How items are found | Main limitation |
|---|---|---|
| Generative recommendation | Decodes item codes from the user context | High decoding cost and ID maintenance |
| Two-tower retrieval | Dot product plus nearest-neighbor index | Shallow user-item interaction |
| Sequential recommendation | Scores all items from a sequence encoder | Softmax cost grows with catalog size |
| Matrix factorization | Latent factors of the user-item matrix | Ignores order and content |
| LLM-based recommendation | Prompts a language model with history | Hallucinated items and latency |
Sequential models such as SASRec and BERT4Rec are close relatives: they also read ordered histories, but they score items discriminatively rather than generating identifiers, so they are often used as the baseline that generative systems must beat.
Training and serving in practice
Training follows next-item prediction with teacher forcing: the model sees a user history and learns to output the code sequence of the item that actually came next. Cross-entropy loss over code tokens replaces the sampled softmax and negative sampling used in many retrieval models, although auxiliary contrastive losses are sometimes added.
Serving is where the main engineering cost appears. Beam search multiplies computation by the beam width, and latency budgets in feeds or e-commerce pages are often tens of milliseconds. Teams use key-value caching, shorter code lengths, smaller decoders and batching to fit the budget.
The following decision table shows when generative recommendation is a reasonable choice and when an established stack is the safer option.
| Situation | Recommended choice | Reason |
|---|---|---|
| Large catalog with rich item content | Generative model with semantic IDs | Shared prefixes help long-tail items |
| Very strict latency budget | Two-tower retrieval plus ranker | Index lookup is cheaper than beam search |
| Rapidly changing catalog | Hybrid with retrieval fallback | New items need codes before generation |
| Long, rich interaction histories | Generative sequential model | Deep context modeling pays off |
| Small catalog and little data | Matrix factorization or SASRec | Generative gains rarely justify the cost |
Limitations and common mistakes
Generative recommendation is promising but not a universal replacement for retrieval. Reported gains depend heavily on datasets, tokenizer quality and how fairly baselines were tuned, and offline improvements do not always survive online A/B tests.
- Unconstrained decoding produces identifiers that match no item, so a validity trie or post-filter is required.
- Stale semantic IDs appear when item content changes but codes are not refreshed, which silently degrades relevance.
- Popularity bias can be amplified because frequent code prefixes dominate beam search.
- Low diversity results when beams converge on one prefix; diverse beam search or re-ranking helps.
- Weak baselines in experiments exaggerate gains; compare against a well-tuned SASRec or two-tower system.
A further risk is the recommendation feedback loop: a model that generates from its own past exposures can narrow what users see over time, so exploration and beyond-accuracy metrics such as catalog coverage remain essential.
The simple explanation
Most recommendation engines work like a librarian who pulls a few hundred books off the shelves and then sorts them. Generative recommendation works differently. Its librarian has memorized a short code for every book. It simply writes down the codes of the books you will probably enjoy next.
Similar books get similar codes. That lets the system guess well even for rare items. What it learned about one book carries over to its neighbors. The catch is cost. Writing out codes piece by piece takes computing power, and the code list must stay current as the catalog changes.
How it is used in practice
A video streaming service with about two million titles wants better recommendations for niche documentaries. Its existing stack uses a two-tower model to fetch 500 candidates and a ranker to order them, but long-tail titles rarely reach the candidate set.
The team builds a generative recommender: each title receives a four-level semantic ID derived from its description, genre tags and trailer embedding, and a Transformer learns to generate the next title from each viewer’s last 200 interactions. Beam search with 50 beams and a validity trie produces candidates that are merged with the two-tower results before ranking.
In a four-week online test, the hybrid system raised viewing of long-tail titles and modestly improved completed views, while latency stayed within budget thanks to caching. The team keeps the old retrieval path as a fallback for newly added titles until their codes are assigned.