Position bias

On this page 7

Position bias is the tendency of users to interact with items placed higher or more prominently in a list, grid or carousel, largely independent of how relevant those items are. In recommender systems, it means that clicks and other implicit feedback reflect where an item was shown as well as whether the user liked it.

Because modern ranking models are trained on logged interactions, position bias is not only a measurement nuisance. Left uncorrected, it feeds a self-reinforcing loop in which items ranked high collect more clicks, are judged more relevant and keep their top spots, while potentially better items stay buried.

Why users favor top positions

Position bias arises because users cannot examine every item with equal attention. Eye-tracking studies on search and recommendation pages consistently show that attention concentrates on the first few positions and decays quickly further down, and that many users stop scrolling once they find something acceptable.

A second cause is trust bias: users assume the system placed the best items first, so they click higher results even when a lower one would serve them better. Layout adds further effects, because items above the fold, in the left column of a grid or in the first slot of a carousel receive more exposure.

The standard way to formalize the effect is the examination hypothesis: an item is clicked only if it is examined and judged relevant. Under this assumption, the click probability equals the examination probability at that position multiplied by the relevance probability of the item.

Click models that describe position bias

Click models are probabilistic descriptions of how users scan a ranked list, and they determine how position bias is estimated and corrected. Each model makes different assumptions about examination.

The table below compares the most common click models used in recommendation and search.

Click modelCore assumptionTypical use
Position-based modelExamination depends only on positionPropensity estimation for debiasing
Cascade modelUser scans top-down and stops at first clickSingle-click search results
Dependent click modelUser may continue after a clickFeeds with several clicks per session
Trust bias modelPerceived relevance is inflated at top positionsCorrecting both examination and trust
Grid modelExposure depends on row and columnTwo-dimensional catalogs and streaming home pages
Common click models and their assumptions

No single model is correct for every interface. A vertical product list on a phone, a carousel and a television home screen produce different exposure curves, so the model must match the actual layout.

Measuring position bias

Position bias is measured by estimating the propensity, the probability that a user examines each position. The most reliable approach is a controlled intervention that changes positions independently of relevance.

The following table compares the main estimation methods.

MethodHow it worksMain drawback
Full randomizationShuffles top results for a small traffic shareVisibly worse experience for that share
Pairwise swapSwaps adjacent positions at randomMeasures only relative propensities
Intervention harvestingUses natural ranking changes across model versionsNeeds several logged rankers
Expectation-maximizationFits relevance and propensity jointly from logsSensitive to model assumptions
Eye tracking or surveysObserves attention directlySmall samples and lab conditions
Methods for estimating position propensities

Propensities typically fall steeply over the first few slots and then flatten. The exact curve depends on device, layout, query intent and user population, so teams should re-estimate propensities after any interface change.

Correcting position bias in ranking models

Correction methods aim to train a model on relevance rather than exposure, so that the learned scores do not simply reproduce the old ranking. They belong to the broader field of unbiased learning to rank and counterfactual learning.

Inverse propensity scoring reweights each click by the inverse of the examination probability at its position, so a click at position eight counts more than a click at position one. It is unbiased when propensities are correct, but small propensities create high variance, which practitioners limit by clipping weights.

A second family treats position as an input feature during training and sets it to a fixed value at serving time. Variants route position through a separate shallow tower whose output is added to the relevance score, so that the main network learns relevance while the auxiliary component absorbs the positional effect.

The decision table below matches common situations with a suitable correction strategy.

SituationRecommended strategyReason
Reliable propensities from randomizationInverse propensity scoring with clippingUnbiased with controlled variance
No randomization budgetPosition feature or shallow towerLearns the bias from logged data
Strong trust effectsAffine correction of clicksHandles inflated clicks at the top
Offline evaluation of a new rankerCounterfactual estimatorsAvoids rewarding the old ranking
Grid or carousel layoutRow and column propensitiesExposure is two-dimensional
Choosing a position bias correction strategy

How position bias relates to other recommendation biases

Position bias is one of several exposure-related biases in recommendation, and it frequently amplifies the others. It is distinct from popularity bias, which favors items with many past interactions, but the two reinforce each other because popular items tend to occupy top positions.

  • Selection bias: users only give feedback on items they were shown at all.
  • Popularity bias: frequently consumed items dominate training data and rankings.
  • Presentation bias: thumbnails, badges and visual size change attention beyond position.
  • Recommendation feedback loop: biased logs train biased models that produce biased logs.
  • Filter bubble: narrowing exposure over time limits diversity and serendipity.

A frequent mistake is to report offline accuracy on logged clicks as proof that a new model is better. Such metrics reward models that imitate the production ranking, because the logged clicks themselves were shaped by its positions.

In plain words

People tend to pick what they see first. On a shopping app or a streaming home page, the items at the top get more clicks simply because they are easier to notice, not always because they are better.

This becomes a problem when a recommendation engine learns from those clicks. It may conclude that the top items are the best ones and keep showing them first. Correcting position bias means separating what people liked from what they merely saw first, so that good items lower in the list get a fair chance.

A real-world example

An online grocery service ranks twenty product suggestions on its mobile home screen. Analysts notice that the first slot receives several times more taps than the fifth slot, even for products of similar quality, and that the ranking barely changes between model updates.

The team randomizes the order of the top ten suggestions for one percent of sessions over two weeks and estimates a propensity for each slot. They then retrain the ranking model with inverse propensity weights clipped at a maximum of 10 and compare it with the old model in an online A/B test.

The corrected model promotes several seasonal and niche products that previously sat below the fold. The test shows higher add-to-basket rates and broader catalog coverage, and the team adopts quarterly propensity re-estimation whenever the home screen layout changes.