Recommendation diversity

On this page 7

Recommendation diversity is the degree to which items in a recommendation list differ from one another in content, category, style or audience, rather than repeating near-identical choices. In Recommender systems, it is one of the main beyond-accuracy metrics, used alongside relevance to judge whether a list is genuinely useful to the person who sees it.

A model optimized only for click-through or rating accuracy tends to return clusters of very similar items, because each one individually scores high. Diversity counters this redundancy. It covers more of a user’s possible interests, reduces the risk that a whole list misses, and helps limit popularity bias and the filter bubble effect.

How recommendation diversity is defined and measured

Recommendation diversity is measured by comparing items with a pairwise dissimilarity function, usually one minus the cosine similarity of item embeddings or a distance between category labels. The most common metric is intra-list diversity (ILD), the average dissimilarity across all pairs of items in one list.

Diversity can be assessed at different levels, and a system can score well on one while failing another. The table below lists the main measures and what each captures.

MeasureWhat it capturesLevel
Intra-list diversityAverage dissimilarity of items in one listIndividual list
Subtopic coverageShare of a user’s interest categories representedIndividual list
Aggregate diversityNumber of distinct items recommended across all usersWhole system
Gini coefficient of exposureHow unevenly impressions spread across the catalogWhole system
Temporal diversityHow much lists change between visitsOver time
Alpha-nDCGRelevance discounted for redundant subtopicsRanked list
Main measures of recommendation diversity

The choice of similarity function determines the result. Two films with different genres but the same director may count as diverse under genre labels and as redundant under an embedding learned from viewing behavior. Teams should state which notion of difference they measure.

Diversity is one of several beyond-accuracy properties, and they are frequently conflated. Each answers a different question about the list or the system, as the following comparison shows.

ConceptCore questionDepends on user history
DiversityAre the recommended items different from each otherNot necessarily
NoveltyIs the item unknown or unpopular for this userYes
SerendipityIs the item surprising and still relevantYes
Catalog coverageHow much of the catalog is ever recommendedNo
Recommendation fairnessAre providers or groups exposed equitablyNot necessarily
Diversity and neighboring concepts

A list can be diverse yet unoriginal, for example ten blockbusters from ten genres, which scores high on intra-list diversity but low on novelty. Conversely, ten obscure jazz albums may be novel while being almost identical to one another.

Methods for increasing diversity

Most production systems add diversity in the final re-ranking stage of multi-stage ranking, after candidate generation and scoring. Re-ranking is cheap, model-agnostic and easy to tune with a single relevance-diversity trade-off parameter.

The best-known method is maximal marginal relevance (MMR), which builds the list greedily: each next item maximizes a weighted sum of its relevance and its dissimilarity to the items already chosen. Determinantal point processes (DPP) pursue the same goal with a probabilistic model that rewards sets whose item vectors span a larger volume.

MethodStageStrengthLimitation
Maximal marginal relevanceRe-rankingSimple and tunableGreedy, ignores global optimum
Determinantal point processRe-rankingPrincipled set-level diversityNeeds a good kernel and more compute
Category quotas or slottingRe-rankingPredictable and explainableRigid, depends on clean taxonomy
xQuAD and subtopic methodsRe-rankingCovers explicit user intentsRequires intent or aspect labels
Multi-objective trainingModel trainingDiversity built into the scoreHarder to tune and evaluate
Exploration policiesServingGathers feedback on unseen itemsShort-term engagement cost
Common diversification methods

A practical rule is to diversify candidate generation as well. If the candidate pool contains only one kind of item, no re-ranker can produce a varied list, so systems often merge candidates from several retrieval sources.

Balancing diversity against relevance

The central trade-off is that more diversity usually lowers offline accuracy metrics such as precision or nDCG, because diverse items are by definition less similar to the user’s strongest past signals. Online, however, moderate diversification often improves long-term engagement and retention, which offline metrics do not capture.

The right level depends on the product surface and the user’s intent, as the decision table below summarizes.

ContextSuggested diversityReason
Home page or discovery feedHighIntent is unclear, broad coverage reduces misses
Search results with a precise queryLowThe user has stated a narrow need
Similar items on a product pageLow to moderateAlternatives should stay comparable
Music or video autoplayModerateAvoid fatigue without breaking the mood
New or cold-start usersHighDiverse lists reveal preferences quickly
Choosing the diversity level by context

Common mistakes and limitations

Diversity work fails most often when teams optimize a metric that users do not perceive. Embedding-based dissimilarity may not match what people consider different, so user studies or online experiments should validate any offline gain.

  • Diversifying with irrelevant items: variety that includes poor matches lowers trust faster than redundancy does.
  • Measuring only intra-list diversity: each list may be varied while all users still receive the same popular items, so aggregate diversity must be checked too.
  • One fixed trade-off for everyone: users differ in appetite for variety, and personalized diversity weights usually work better.
  • Ignoring the feedback loop: low-diversity lists produce narrow interaction data, which trains the next model to be even narrower.
  • Confusing diversity with fairness: varied lists do not guarantee equitable exposure for providers.

In simple terms

Imagine asking a friend for ten film suggestions and getting ten nearly identical action sequels. Each one might suit you, but the list as a whole is not very helpful. Recommendation diversity is about making sure a list offers real variety while still fitting your taste.

A diverse list is like a good restaurant menu: it has dishes you already love, plus a few different options you might enjoy. If the system guesses wrong about one item, the rest of the list still has a chance. Too much variety, however, fills the list with things you do not want, so the goal is a sensible balance.

A practical example

An online bookstore notices that readers who buy one crime novel receive a home page filled almost entirely with crime fiction by the same few authors. Click-through stays high at first, but many customers stop visiting after a few weeks, and sales concentrate on a small share of the catalog.

The engineering team adds an MMR re-ranking step to the top 20 positions, using genre and author embeddings as the dissimilarity measure, and tests three trade-off weights in an A/B experiment over six weeks. They also merge candidates from a second retrieval source based on recent purchases across categories.

The middle setting reduces offline precision slightly but raises the number of distinct genres per page from about two to five. Over the test period, return visits and the share of catalog titles sold both increase, while the heaviest crime readers, who received a lower personal diversity weight, show no drop in purchases.