Reranking is the second stage of a retrieval pipeline in which a more accurate but slower model rescores a short list of candidate passages and reorders them by relevance to the query. In artificial intelligence systems that answer questions from documents, it sits between first-stage retrieval, which casts a wide and cheap net, and the language model or user that consumes only the top few results.
The idea rests on a simple division of labor: recall is cheap at scale, precision is expensive. A vector index or keyword engine can scan millions of chunks in milliseconds, but it scores each one with a coarse signal. A reranker spends far more computation per candidate, so it is applied only to the top 20 to 200 hits, where its judgment changes what the user actually sees.
How reranking works in a retrieval pipeline
A reranker receives the query and a candidate list from the first stage and returns a new relevance score for every candidate. The pipeline then sorts by that score and keeps the top k results, typically 3 to 10 for retrieval-augmented generation (RAG) or 10 to 20 for a search results page.
The most common design is the cross-encoder. Unlike a bi-encoder, which embeds the query and the document separately and compares the vectors, a cross-encoder reads the query and passage together in one input. Every token of the query can attend to every token of the passage, so the model can detect negation, exact entity matches and whether the passage actually answers the question rather than merely sharing its vocabulary.
The following table shows where reranking fits among the stages of a typical pipeline.
| Stage | Typical input size | Typical method | Main goal |
|---|---|---|---|
| First-stage retrieval | Millions of chunks | BM25, vector search or hybrid | High recall at low latency |
| Reranking | 20 to 200 candidates | Cross-encoder or LLM scorer | High precision at the top |
| Filtering and truncation | Reranked list | Score threshold, top k, deduplication | Fit the context budget |
| Consumption | 3 to 10 passages | Language model or result page | Answer or display |
Types of rerankers and their trade-offs
Rerankers differ mainly in how they compute relevance and therefore in cost and quality. The choice depends on latency budget, candidate count, language coverage and whether scores must be calibrated across queries.
| Reranker type | How it scores | Relative cost | Strengths and weaknesses |
|---|---|---|---|
| Cross-encoder | Joint encoding of query and passage | Moderate | Strong precision; input length limited |
| Late-interaction model | Token-level vector matching, as in ColBERT | Low to moderate | Precomputable; larger index storage |
| LLM pointwise scorer | Prompted relevance grade per passage | High | Flexible criteria; slow and costly |
| LLM listwise reranker | Reorders a whole list in one prompt | High | Sees context between candidates; position bias |
| Heuristic reranker | Rules on recency, source or metadata | Very low | Transparent; ignores meaning |
Pointwise methods score each passage independently, pairwise methods decide which of two passages is better, and listwise methods order a full set at once. Listwise prompting with a large language model can capture comparisons between candidates, but it is sensitive to the order in which candidates are presented and usually needs a sliding window for lists longer than about 20 items.
Reranking compared with related retrieval techniques
Reranking is often confused with the retrieval methods it complements. Semantic search and vector similarity decide which candidates enter the shortlist; reranking decides their final order. It cannot recover a relevant document that the first stage missed, which is why recall at the first stage remains the hard ceiling of the whole system.
| Concept | What it decides | Operates on | Relation to reranking |
|---|---|---|---|
| Vector similarity | Closeness of two embeddings | Whole index | Supplies first-stage candidates |
| Semantic search | Which documents match by meaning | Whole index | Often the stage before reranking |
| Chunking | How documents are split | Source documents | Shapes what the reranker can judge |
| RAG | What context the model receives | Final passages | Consumes the reranked top k |
| Query rewriting | How the query is phrased | User input | Improves both stages upstream |
Chunking quality matters a great deal. A reranker judges each chunk in isolation, so a chunk that loses its heading or splits an answer across a boundary may be scored as irrelevant even when its document is correct. Adding the document title or section path to each chunk before reranking is a cheap and effective fix.
Configuring reranking in practice
Three parameters govern most deployments: the candidate depth passed to the reranker, the final top k and an optional score threshold. Deeper candidate lists raise the chance that the right passage is present but increase latency roughly linearly, because each candidate needs its own forward pass in a cross-encoder.
| Situation | Recommended setting | Reason |
|---|---|---|
| Interactive chat with tight latency | Rerank 20 to 50, keep 3 to 5 | Keeps added delay small |
| Long technical documentation | Rerank 100 to 200, keep 5 to 10 | First stage misses fine distinctions |
| Multilingual corpus | Multilingual cross-encoder | Monolingual models misjudge other languages |
| Answer may not exist | Apply a calibrated score threshold | Allows the system to say nothing matched |
| Strict compliance sources | Combine model score with metadata rules | Relevance alone ignores authority |
Teams should measure the reranker separately from the rest of the pipeline, using metrics such as nDCG@10, MRR or recall@k on a labeled set of real queries. A reranker that improves benchmark scores on public data can still underperform on a domain with specialized terminology, so domain evaluation comes before deployment.
Limitations and common mistakes
- Treating reranking as a recall fix. If the first stage returns no relevant candidate, no reranker can help.
- Exceeding the input limit. Many cross-encoders accept roughly 512 tokens per pair, so long chunks are silently truncated.
- Comparing raw scores across queries. Cross-encoder logits are rarely calibrated, so a fixed threshold needs validation.
- Ignoring added latency. Reranking 200 candidates on a CPU can cost more time than the rest of the pipeline.
- Duplicate passages crowding the top. Near-identical chunks should be deduplicated after reranking.
A subtler problem is distribution shift. Rerankers trained on web search questions learn what a good answer looks like in that setting and may undervalue tables, code or legal clauses. Fine-tuning on a few thousand labeled query-passage pairs from the target domain usually corrects this.
A plain-language summary
Imagine asking a librarian for help. A quick assistant first pulls fifty books that mention your topic from the shelves. Then an expert reads the relevant pages of each one and puts the five most useful books on top of the pile. Reranking is that expert second look.
The first step is fast but rough, and the second step is slow but careful. Because the careful step only looks at a small pile, the whole process stays quick while the best answers end up first.
An example from practice
A support team runs an assistant over 40,000 help-center chunks. Customers ask questions such as how to cancel a plan without losing stored invoices. Vector search alone often returns chunks about canceling a plan or about invoices, but rarely the one passage that covers both.
The team adds a cross-encoder that rescores the top 50 vector hits and passes the best 5 to the language model. On a labeled set of 300 real customer questions, the correct passage now appears in the top 5 far more often, and answers that mixed up unrelated policies become rare. The added delay is a few hundred milliseconds on a GPU, which the team accepts because answer accuracy improved noticeably while the index and the language model stayed unchanged.