Learning to rank (LTR), also called machine-learned ranking, is the family of supervised machine learning methods that train a model to order a set of items by relevance to a user, query or context, rather than to predict an isolated score for each item. In recommender systems, learning to rank usually powers the final ranking stage, where a few hundred candidates are sorted into the short list a person actually sees.
The approach matters because users interact almost entirely with the top positions of a list. A model that predicts ratings accurately but places the best items in positions eight to twelve delivers little value, so learning to rank optimizes the quality of the ordering itself, measured with list-aware metrics such as NDCG and mean reciprocal rank.
How learning to rank models order items
A learning to rank model learns a scoring function that maps a feature vector for each user-item or query-item pair to a real number, and the list is then sorted by that number. Training data consists of groups of items that share one context, such as one search query or one user session, each item carrying a graded relevance label or an implicit signal such as a click, purchase or watch time.
What distinguishes LTR from ordinary regression is the loss function. Instead of penalizing the distance between a predicted and a true rating, ranking losses penalize mistakes in relative order: a relevant item scored below an irrelevant one, or a highly relevant item pushed out of the top positions. The model therefore spends its capacity where ranking quality is decided.
Typical input features fall into a few families, and the table below summarizes them with examples from a retail recommender.
| Feature family | Examples | What it captures |
|---|---|---|
| Item features | Category, price, age, popularity | Intrinsic appeal of the item |
| User features | Tenure, preferred brands, device | Stable user preferences |
| Interaction features | Past clicks on the brand, embedding similarity | Fit between user and item |
| Context features | Time of day, page type, session length | Situation of the request |
| Upstream scores | Candidate generator score, retrieval rank | Signals from earlier stages |
Pointwise, pairwise and listwise approaches
Learning to rank methods are classified into three approaches according to how many items the loss function looks at simultaneously. The choice affects training cost, robustness to noisy labels and how closely the objective matches the evaluation metric.
| Approach | Loss looks at | Representative methods | Main trade-off |
|---|---|---|---|
| Pointwise | One item at a time | Logistic regression on clicks, McRank | Simple, but ignores relative order |
| Pairwise | Pairs of items | RankNet, RankSVM, Bayesian personalized ranking | Learns order, but treats all pairs equally |
| Listwise | The whole list | ListNet, ListMLE, LambdaMART | Closest to metrics, higher training cost |
LambdaMART, which combines gradient-boosted decision trees with lambda gradients derived from NDCG changes, has been a strong production baseline for well over a decade. Its lambda trick scales each pairwise gradient by how much swapping the two items would change the metric, so errors at the top of the list cost more than errors near the bottom. Neural rankers built on deep networks or transformers now dominate when feature sets are large and include raw text or sequences, but tree ensembles remain competitive on tabular features.
Metrics that evaluate ranked lists
Ranking quality is measured with position-aware metrics that reward relevant items placed near the top. Most of them are computed at a cutoff k, written as @k, which reflects the number of slots a user realistically inspects.
| Metric | What it measures | Best suited to |
|---|---|---|
| NDCG@k | Graded gain discounted by log of position | Multi-level relevance labels |
| MRR | Inverse rank of the first relevant item | Tasks with one correct answer |
| MAP | Mean precision at each relevant position | Binary relevance with many hits |
| Recall@k | Share of relevant items in the top k | Candidate generation stages |
| Precision@k | Share of top k that is relevant | Short fixed carousels |
Offline metrics are necessary but not sufficient. Because logged data reflects what the previous system chose to show, offline gains often shrink online, and teams confirm improvements with A/B tests or interleaving experiments that compare two rankers on live traffic.
Learning to rank in a multi-stage recommender
In a production multi-stage ranking architecture, learning to rank occupies the expensive, precise end of the pipeline. Candidate generation, often a two-tower model with approximate nearest neighbor search, reduces millions of items to hundreds; a lightweight pre-ranker may cut them to dozens; the main LTR model then applies rich cross features; and a re-ranking step adjusts the result for diversity, business rules and fairness.
The comparison below shows how learning to rank relates to neighboring techniques in recommender systems.
| Technique | Primary output | Relationship to LTR |
|---|---|---|
| Rating prediction | Estimated score per item | Pointwise special case without list objective |
| CTR prediction | Click probability per item | Often used as a pointwise ranking signal |
| Matrix factorization | Latent user and item vectors | Supplies features or candidates |
| Candidate generation | Broad set of plausible items | Feeds the ranking model |
| Re-ranking | Final adjusted order | Post-processes LTR output |
Modern rankers frequently optimize several objectives at once, such as click probability, purchase probability and long-term satisfaction, and combine them into a single ranking score with tuned weights. This multi-objective setup is a practical extension of learning to rank rather than a separate discipline.
Limitations and common mistakes
Most failures of learning to rank come from the training data, not the algorithm. Implicit feedback is biased: users click items in top positions partly because those positions are visible, which is known as position bias, and they never click items the system never showed.
- Ignoring position bias teaches the model to reproduce the old ranking; unbiased learning to rank corrects it with inverse propensity weighting or click models.
- Treating non-clicks as negatives confuses unseen items with disliked items; careful negative sampling and impression logs reduce the error.
- Optimizing clicks alone rewards clickbait; adding dwell time, conversions or satisfaction labels aligns the objective with real value.
- Training-serving skew occurs when features are computed differently offline and online, silently degrading the live ranking.
- Feedback loops narrow exposure over time and reinforce popularity bias unless exploration or diversity constraints are added.
Learning to rank also inherits the cold start problem: new items without interaction history receive weak feature values and are ranked low, so systems reserve exploration slots or rely on content-based features to give them a fair chance.
In plain words
Learning to rank teaches a computer to put the best options first. Instead of asking how much someone will like each item on its own, the model asks which item should come before which other item.
Think of a librarian arranging a shelf of recommendations for one reader. The librarian does not need to grade every book precisely; the job is to make sure the books the reader will love sit at eye level. Because people rarely look past the first few results, getting the top of the list right is what matters most.
A real-world example
A mid-sized online fashion retailer shows a carousel of 12 recommended products on each product page. Its first system ranked items by predicted click probability from a pointwise logistic regression, and the team noticed that most clicks landed on the first three slots regardless of quality.
The data science team trained a LambdaMART model optimized for NDCG@12, using about 150 features covering item attributes, user history and session context. They corrected position bias with inverse propensity weights estimated from a small randomized traffic slice, where the top slots were shuffled for one percent of visitors.
Offline NDCG@12 improved by several points, and a four-week A/B test showed a measurable rise in add-to-cart rate from the carousel, with the largest gains for less popular products that the old model had buried. The team kept the randomized slice running to keep propensity estimates current.