Negative sampling

On this page 7

Negative sampling is a training technique that pairs each observed user-item interaction with a small number of items the user did not interact with, so that a model learns to score relevant items above irrelevant ones without processing the entire catalog. In recommender systems, one of the most applied areas of artificial intelligence, it is the standard way to learn from implicit feedback such as clicks, plays and purchases, where only positive signals are recorded.

The technique solves a practical problem: a catalog may contain millions of items, and computing a loss over all of them for every training example is far too expensive. Negative sampling replaces that full computation with a cheap, randomized approximation, and the way negatives are chosen strongly shapes what the model learns.

Why recommenders need sampled negatives

Recommenders need sampled negatives because implicit feedback has no explicit negative labels. A log shows that a user watched a film, but the absence of a view does not mean dislike: the user may simply never have seen the title. Treating every unobserved item as a confirmed negative is both computationally prohibitive and statistically wrong.

Negative sampling resolves both issues at once. Each positive example is contrasted with a handful of unobserved items treated as probable negatives, which keeps training cost proportional to the number of interactions rather than to the size of the user-item matrix. Because most unobserved items really are irrelevant to a given user, the noise introduced by occasional false negatives is usually tolerable.

How negative sampling works during training

Negative sampling works by drawing, for each positive pair, k negative items from a sampling distribution and optimizing a loss that pushes the positive score up and the negative scores down. The number k typically ranges from 1 to a few hundred, depending on the loss and the model.

The choice of loss function determines how positives and negatives interact. The following table compares the losses most often combined with sampled negatives.

LossHow negatives are usedTypical setting
Binary cross-entropyEach negative is a separate 0 labelNeural collaborative filtering, CTR models
Bayesian personalized rankingOne negative per positive in a pairwise comparisonMatrix factorization on implicit data
Sampled softmaxNegatives form the denominator of a softmaxTwo-tower retrieval, sequential models
Hinge or margin lossPositive must beat negatives by a marginMetric learning, item embedding
Loss functions commonly used with negative sampling

With sampled softmax, the logits of sampled negatives must be corrected by subtracting the log of their sampling probability. Without this logQ correction, items that are sampled often, usually popular ones, are penalized too strongly and the learned scores become biased.

Sampling strategies and their trade-offs

The sampling strategy decides which unobserved items count as negatives, and it is the main lever that practitioners tune. Easy negatives teach coarse distinctions, while hard negatives teach the fine distinctions that matter at the top of a ranked list.

StrategyHow negatives are chosenStrengthWeakness
Uniform samplingRandom items from the catalogSimple and unbiased over itemsMostly trivial negatives
Popularity-based samplingProbability proportional to frequency, often raised to 0.75Counters popularity bias in scoresNeeds logQ correction
In-batch negativesPositives of other examples in the batchNearly free to computeSkewed toward popular items
Hard negative miningHigh-scoring items under the current modelSharpens top-N rankingRisk of false negatives
Mixed negative samplingIn-batch plus uniform negativesBalances bias and coverageExtra tuning effort
Main negative sampling strategies

In-batch negatives dominate large-scale two-tower training because the item embeddings are already computed, so each batch of size B provides B minus one negatives per example at almost no cost. The trade-off is that batch contents follow the interaction distribution, so popular items appear as negatives far more often than rare ones.

Negative sampling is one of several ways to handle unobserved data, and its main alternatives either weight all missing entries or avoid sampling altogether. The comparison below places it among those neighbors.

ApproachTreatment of unobserved itemsCost per epochTypical model
Negative samplingSmall random subset as negativesProportional to interactionsDeep and factorization models
Weighted full matrixAll missing entries with low confidenceEfficient only with special solversAlternating least squares
Full softmaxEvery item in the denominatorProportional to catalog sizeSmall catalogs only
Explicit negativesOnly items the user rejectedDepends on available signalsRating prediction, skip data
Negative sampling versus alternative treatments of unobserved items

Alternating least squares with confidence weights can use every missing entry efficiently, but only for linear factorization. Once the model includes deep towers, sequential encoders or side features, sampling becomes the practical choice.

Common pitfalls and how to avoid them

The most common pitfalls involve false negatives, biased sampling and misleading evaluation. Each of them can make offline metrics look good while the deployed recommender behaves poorly.

  • False negatives: hard negatives often include items the user would like but has not discovered; filtering known positives and capping hardness reduces the damage.
  • Uncorrected sampling bias: skipping the logQ correction with popularity or in-batch sampling distorts scores for head and tail items.
  • Sampled evaluation: ranking the true item against only 100 random negatives can reverse model comparisons; full-catalog ranking metrics are more reliable.
  • Too few negatives: a single negative per positive slows convergence for softmax-style losses and leaves fine distinctions unlearned.
  • Exposure blindness: items never shown to a user are not true negatives; position bias and exposure data, when available, give better signals.

A sound default is to start with in-batch negatives plus logQ correction, add a share of uniform negatives for catalog coverage, and introduce hard negatives gradually once the base model is stable.

In simple terms

A recommendation service usually knows only what people did choose, not what they disliked. To learn good taste, a model still needs examples of what a person probably does not want.

Negative sampling supplies those examples by picking a few random items the person skipped and telling the model to rank the chosen item above them. It is similar to teaching someone to recognize a friend in a crowd by showing the friend next to a few strangers, instead of next to every person in the city. Choosing those strangers carefully matters: lookalikes teach more than obvious mismatches, but a lookalike might secretly be another friend.

A practical example

A music streaming team trains a two-tower retrieval model on 500 million listening events across a catalog of 30 million tracks. Computing a softmax over every track for each event is impossible within the daily training window, so the team uses in-batch negatives with a batch size of 4,096 and applies the logQ correction based on track play frequency.

The first version recommends mostly mainstream hits, so the team adds 1,024 uniformly sampled tracks per batch as extra negatives, which gives obscure titles a fair chance of being contrasted. After that change, offline recall at 100 measured over the full catalog improves noticeably, and an online test shows more listening to long-tail artists without a drop in overall session length. The team keeps sampled-candidate evaluation out of its reports because it had previously overstated the gains of an earlier model.