Candidate generation, also called candidate retrieval, is the first stage of a multi-stage recommendation pipeline, in which the system selects a few hundred to a few thousand potentially relevant items from a catalog that may contain millions or billions of entries. In recommender systems, it determines what the later ranking models are even allowed to consider, so it sets the upper bound on recommendation quality.
The stage exists because precise ranking models are too expensive to score every item for every request. Candidate generation therefore trades precision for speed: it uses cheap similarity computations, precomputed indexes and simple heuristics to return a broad, high-recall set within a strict latency budget, typically a few tens of milliseconds.
How candidate generation fits into multi-stage ranking
Large production recommenders split the work into a funnel of progressively more expensive stages. Candidate generation sits at the wide top of the funnel, followed by an optional pre-ranking step, a heavy ranking model such as a CTR prediction network, and a final re-ranking layer that applies business rules, diversity and freshness constraints.
The following table shows the typical scale and purpose of each stage in such a funnel; exact numbers vary by product and catalog size.
| Stage | Items in | Items out | Main objective |
|---|---|---|---|
| Candidate generation | Millions to billions | Hundreds to thousands | High recall at low cost |
| Pre-ranking | Thousands | Hundreds | Cheap filtering with a light model |
| Ranking | Hundreds | Tens | Precise scoring of engagement or value |
| Re-ranking | Tens | The final list | Diversity, fairness and business rules |
Because every later stage only reorders what it receives, an item that candidate generation misses can never be shown. This is why teams evaluate retrieval mainly by recall at K, the share of items a user eventually engages with that appear in the top K retrieved candidates, rather than by ranking metrics such as NDCG.
Main candidate generation methods
Modern systems rarely rely on a single source. Instead they run several retrieval channels in parallel, often called recall paths or candidate sources, and merge their outputs. Each channel captures a different signal, such as long-term taste, recent behavior, popularity or item similarity.
The table below compares the most common candidate sources and the situations in which each performs well.
| Method | How it retrieves items | Strength | Weakness |
|---|---|---|---|
| Two-tower embedding retrieval | Nearest neighbors of a user vector in item embedding space | Personalized and scalable | Needs training and an ANN index |
| Item-to-item co-occurrence | Items often consumed with recent items | Strong for related content | Weak for new items |
| Matrix factorization | Dot product of learned user and item factors | Captures latent taste | Retrained in batch, less fresh |
| Content-based retrieval | Items with similar text, image or metadata features | Handles cold start items | Tends toward narrow results |
| Popularity and trending lists | Globally or regionally popular items | Robust fallback for new users | Reinforces popularity bias |
| Graph-based retrieval | Random walks or LightGCN on the user-item graph | Exploits multi-hop relations | Graph maintenance cost |
The two-tower model has become the dominant learned approach. One neural tower encodes the user and context, another encodes the item, and relevance is the dot product or cosine similarity of the two vectors. Because item vectors can be precomputed, retrieval reduces to a fast approximate nearest neighbor (ANN) search over an index built with methods such as HNSW graphs or product quantization.
Training retrieval models with negative sampling
Learned candidate generators are trained mostly on implicit feedback such as clicks, watches and purchases, so they see positive examples but no explicit negatives. The model must therefore be taught what irrelevant looks like through negative sampling, and the choice of negatives strongly shapes retrieval quality.
This table summarizes the main negative sampling strategies used for retrieval models.
| Strategy | Source of negatives | Effect on the model |
|---|---|---|
| Random negatives | Uniformly sampled catalog items | Easy to learn, weak discrimination |
| In-batch negatives | Positives of other users in the same batch | Efficient, but biased toward popular items |
| Hard negatives | Items scored high but not engaged with | Sharper boundaries, risk of false negatives |
| Mixed negatives | Combination of random and in-batch items | Balances popularity bias and efficiency |
In-batch sampling oversamples popular items, so practitioners apply a logQ correction that subtracts the logarithm of each item’s sampling probability from its score. Without it, the model learns to penalize popular items unfairly, which harms recall on mainstream content.
Candidate generation compared with ranking
Candidate generation and ranking solve related but different problems. Retrieval must be computable before the item is known, so user and item representations are usually separate, while ranking can use rich cross features that combine user, item and context, such as how often this user clicked this category at this hour.
- Objective: retrieval optimizes recall over the full catalog; ranking optimizes precision on a short list.
- Features: retrieval relies on separable embeddings; ranking uses interaction features and deep networks such as DeepFM or Deep & Cross Network.
- Latency: retrieval must scan the catalog in milliseconds; ranking can spend more time per item on a small set.
- Freshness: retrieval indexes are often rebuilt periodically; ranking models can react to real-time features.
Newer generative recommendation approaches blur this boundary by producing item identifiers directly with a sequence model, but most production systems still keep an explicit retrieval stage because it is cheaper and easier to control.
Risks and common design mistakes
Because candidate generation defines the space of possible recommendations, its weaknesses propagate through the entire system. A retrieval layer dominated by popularity sources narrows catalog coverage and can strengthen the recommendation feedback loop, in which the system only learns from items it already chose to show.
- Evaluating only end-to-end metrics, which hides whether losses come from retrieval or from ranking.
- Training and serving mismatch, for example computing user features differently online than offline.
- Stale ANN indexes that omit newly added items for hours or days.
- Overlapping channels that return the same items and waste the candidate budget.
- Ignoring exploration, so new items never collect the interactions needed to be retrieved.
Experienced teams therefore reserve a quota for exploration and fresh items, deduplicate across channels before ranking, and track per-channel recall so that each source can be justified by its unique contribution.
Understanding it simply
Think of a librarian helping you choose a book in a library with a million titles. Reading every book to judge it for you would take forever, so the librarian first pulls a cartful of promising titles based on what you liked before, what similar readers borrowed and what is popular this month. Only then does the librarian look closely at that cart and hand you the best few.
Candidate generation is that first quick sweep of the shelves. It does not need to be perfect, but it must not leave out the books you would love, because anything left on the shelf can never be recommended.
Applied in practice
A video streaming service with about two million titles and clips must return a personalized home feed in under 150 milliseconds. Its retrieval layer runs four channels in parallel: a two-tower model that returns 500 items from an ANN index, an item-to-item channel based on the last five watched videos returning 200 items, a trending channel with 100 items and a fresh-content channel with 100 newly uploaded items.
After deduplication, roughly 750 candidates reach a deep ranking model that scores watch probability and expected watch time. When the team added the fresh-content channel and logQ correction to the two-tower model, recall at 500 rose from 41 to 48 percent in offline tests, and new uploads received their first views within hours instead of days. The ranking model was unchanged, which showed that the gain came entirely from better candidate generation.