Candidate generation

Also known as: candidate retrieval

On this page 7

Candidate generation, also called candidate retrieval, is the first stage of a multi-stage recommendation pipeline, in which the system selects a few hundred to a few thousand potentially relevant items from a catalog that may contain millions or billions of entries. In recommender systems, it determines what the later ranking models are even allowed to consider, so it sets the upper bound on recommendation quality.

The stage exists because precise ranking models are too expensive to score every item for every request. Candidate generation therefore trades precision for speed: it uses cheap similarity computations, precomputed indexes and simple heuristics to return a broad, high-recall set within a strict latency budget, typically a few tens of milliseconds.

How candidate generation fits into multi-stage ranking

Large production recommenders split the work into a funnel of progressively more expensive stages. Candidate generation sits at the wide top of the funnel, followed by an optional pre-ranking step, a heavy ranking model such as a CTR prediction network, and a final re-ranking layer that applies business rules, diversity and freshness constraints.

The following table shows the typical scale and purpose of each stage in such a funnel; exact numbers vary by product and catalog size.

StageItems inItems outMain objective
Candidate generationMillions to billionsHundreds to thousandsHigh recall at low cost
Pre-rankingThousandsHundredsCheap filtering with a light model
RankingHundredsTensPrecise scoring of engagement or value
Re-rankingTensThe final listDiversity, fairness and business rules
Typical stages of a recommendation funnel

Because every later stage only reorders what it receives, an item that candidate generation misses can never be shown. This is why teams evaluate retrieval mainly by recall at K, the share of items a user eventually engages with that appear in the top K retrieved candidates, rather than by ranking metrics such as NDCG.

Main candidate generation methods

Modern systems rarely rely on a single source. Instead they run several retrieval channels in parallel, often called recall paths or candidate sources, and merge their outputs. Each channel captures a different signal, such as long-term taste, recent behavior, popularity or item similarity.

The table below compares the most common candidate sources and the situations in which each performs well.

MethodHow it retrieves itemsStrengthWeakness
Two-tower embedding retrievalNearest neighbors of a user vector in item embedding spacePersonalized and scalableNeeds training and an ANN index
Item-to-item co-occurrenceItems often consumed with recent itemsStrong for related contentWeak for new items
Matrix factorizationDot product of learned user and item factorsCaptures latent tasteRetrained in batch, less fresh
Content-based retrievalItems with similar text, image or metadata featuresHandles cold start itemsTends toward narrow results
Popularity and trending listsGlobally or regionally popular itemsRobust fallback for new usersReinforces popularity bias
Graph-based retrievalRandom walks or LightGCN on the user-item graphExploits multi-hop relationsGraph maintenance cost
Common candidate sources and their strengths

The two-tower model has become the dominant learned approach. One neural tower encodes the user and context, another encodes the item, and relevance is the dot product or cosine similarity of the two vectors. Because item vectors can be precomputed, retrieval reduces to a fast approximate nearest neighbor (ANN) search over an index built with methods such as HNSW graphs or product quantization.

Training retrieval models with negative sampling

Learned candidate generators are trained mostly on implicit feedback such as clicks, watches and purchases, so they see positive examples but no explicit negatives. The model must therefore be taught what irrelevant looks like through negative sampling, and the choice of negatives strongly shapes retrieval quality.

This table summarizes the main negative sampling strategies used for retrieval models.

StrategySource of negativesEffect on the model
Random negativesUniformly sampled catalog itemsEasy to learn, weak discrimination
In-batch negativesPositives of other users in the same batchEfficient, but biased toward popular items
Hard negativesItems scored high but not engaged withSharper boundaries, risk of false negatives
Mixed negativesCombination of random and in-batch itemsBalances popularity bias and efficiency
Negative sampling strategies for retrieval training

In-batch sampling oversamples popular items, so practitioners apply a logQ correction that subtracts the logarithm of each item’s sampling probability from its score. Without it, the model learns to penalize popular items unfairly, which harms recall on mainstream content.

Candidate generation compared with ranking

Candidate generation and ranking solve related but different problems. Retrieval must be computable before the item is known, so user and item representations are usually separate, while ranking can use rich cross features that combine user, item and context, such as how often this user clicked this category at this hour.

  • Objective: retrieval optimizes recall over the full catalog; ranking optimizes precision on a short list.
  • Features: retrieval relies on separable embeddings; ranking uses interaction features and deep networks such as DeepFM or Deep & Cross Network.
  • Latency: retrieval must scan the catalog in milliseconds; ranking can spend more time per item on a small set.
  • Freshness: retrieval indexes are often rebuilt periodically; ranking models can react to real-time features.

Newer generative recommendation approaches blur this boundary by producing item identifiers directly with a sequence model, but most production systems still keep an explicit retrieval stage because it is cheaper and easier to control.

Risks and common design mistakes

Because candidate generation defines the space of possible recommendations, its weaknesses propagate through the entire system. A retrieval layer dominated by popularity sources narrows catalog coverage and can strengthen the recommendation feedback loop, in which the system only learns from items it already chose to show.

  • Evaluating only end-to-end metrics, which hides whether losses come from retrieval or from ranking.
  • Training and serving mismatch, for example computing user features differently online than offline.
  • Stale ANN indexes that omit newly added items for hours or days.
  • Overlapping channels that return the same items and waste the candidate budget.
  • Ignoring exploration, so new items never collect the interactions needed to be retrieved.

Experienced teams therefore reserve a quota for exploration and fresh items, deduplicate across channels before ranking, and track per-channel recall so that each source can be justified by its unique contribution.

Understanding it simply

Think of a librarian helping you choose a book in a library with a million titles. Reading every book to judge it for you would take forever, so the librarian first pulls a cartful of promising titles based on what you liked before, what similar readers borrowed and what is popular this month. Only then does the librarian look closely at that cart and hand you the best few.

Candidate generation is that first quick sweep of the shelves. It does not need to be perfect, but it must not leave out the books you would love, because anything left on the shelf can never be recommended.

Applied in practice

A video streaming service with about two million titles and clips must return a personalized home feed in under 150 milliseconds. Its retrieval layer runs four channels in parallel: a two-tower model that returns 500 items from an ANN index, an item-to-item channel based on the last five watched videos returning 200 items, a trending channel with 100 items and a fresh-content channel with 100 newly uploaded items.

After deduplication, roughly 750 candidates reach a deep ranking model that scores watch probability and expected watch time. When the team added the fresh-content channel and logQ correction to the two-tower model, recall at 500 rose from 41 to 48 percent in offline tests, and new uploads received their first views within hours instead of days. The ranking model was unchanged, which showed that the gain came entirely from better candidate generation.