A retriever is the component of an information access pipeline that takes a query and returns a small ranked set of relevant passages from a much larger collection. In the field of retrieval & knowledge within artificial intelligence, the retriever is the first stage of retrieval-augmented generation, where it supplies a language model with the evidence it needs to answer accurately.
The retriever does not generate text and does not judge final correctness. Its job is recall under a latency budget: find the passages that matter, quickly, so that later stages such as a reranker or a generator can work with a manageable amount of context.
How a retriever finds relevant passages
A retriever works by turning both the query and the stored documents into comparable representations and then scoring how well each document matches. The collection is processed ahead of time during ingestion, so that at query time the system only has to encode the query and search an index, rather than read every document.
The process has two phases. In the offline indexing phase, documents are split into chunks, each chunk is converted into a sparse term vector, a dense embedding or both, and the results are stored in an index. In the online query phase, the query is encoded the same way, the index returns the top-k candidates, and optional filters remove items that violate metadata constraints such as date, language or access rights.
The following table summarizes the main steps and what each one controls.
| Stage | What happens | Main design choice |
|---|---|---|
| Chunking | Documents are split into passages | Chunk size and overlap |
| Encoding | Text becomes vectors or term weights | Embedding model or term weighting |
| Indexing | Representations are stored for fast lookup | Exact or approximate index |
| Query encoding | The user query is converted | Same model as the documents |
| Candidate search | The index returns top-k matches | Value of k and similarity metric |
| Filtering | Results are restricted by metadata | Pre-filtering or post-filtering |
Sparse, dense and hybrid retrievers
Retrievers fall into three main families that differ in how they represent meaning. A sparse retriever scores documents by weighted term overlap, typically with BM25 or TF-IDF, and excels at exact matches on names, codes and rare words. A dense retriever uses a bi-encoder embedding model to place queries and passages in a shared vector space, so it can match paraphrases and synonyms that share no words.
Hybrid retrievers run both approaches and merge the ranked lists, often with reciprocal rank fusion. Learned sparse models such as SPLADE and late-interaction models such as ColBERT sit between the families, keeping term-level precision while adding learned semantics. The table compares the families on the properties that usually decide the choice.
| Retriever type | Representation | Strength | Weakness |
|---|---|---|---|
| Sparse (BM25) | Inverted index of weighted terms | Exact terms, no training needed | Misses synonyms and paraphrases |
| Dense (bi-encoder) | One embedding per passage | Semantic similarity | Weak on rare identifiers |
| Learned sparse (SPLADE) | Expanded term weights | Interpretable and semantic | Larger index, model required |
| Late interaction (ColBERT) | One vector per token | Fine-grained matching | High storage cost |
| Hybrid | Sparse and dense lists fused | Robust across query types | Two indexes to maintain |
Retriever versus reranker and generator
The retriever is distinct from the components that follow it, and confusing them leads to poor system design. The retriever must be fast enough to search millions of passages, so it compares precomputed representations. A reranker, usually a cross-encoder, reads the query and each candidate together and is far more accurate, but it is too slow to apply to the whole collection.
This division produces the common retrieve-then-rerank pattern: the retriever returns perhaps 50 to 200 candidates with high recall, and the reranker reorders them so that the best 3 to 10 reach the generator. The generator then grounds its answer in those passages.
| Component | Primary goal | Typical input size | Relative cost per item |
|---|---|---|---|
| Retriever | High recall | Entire collection | Very low |
| Reranker | High precision | Tens to hundreds of candidates | Moderate |
| Generator | Faithful answer | A few passages | High |
Measuring retriever quality
Retriever quality is measured with ranking metrics computed against a set of queries with known relevant passages. The most important metric for a first-stage retriever is recall at k, because a passage that is not retrieved can never be used later, no matter how good the reranker or generator is.
Teams also track mean reciprocal rank and nDCG to see how high the relevant passages appear, and they monitor latency percentiles because retrieval sits on the critical path of every request. Public benchmarks give a rough orientation, but domain-specific evaluation sets are far more predictive, since retriever performance varies sharply between general web text and specialized material such as contracts or source code.
| Metric | What it measures | When it matters most |
|---|---|---|
| Recall at k | Share of relevant passages found in top k | First-stage retrieval |
| Mean reciprocal rank | Position of the first relevant result | Single-answer questions |
| nDCG at k | Graded relevance weighted by position | Multiple relevant passages |
| p95 latency | Slowest typical response time | Interactive applications |
Common retriever failures and fixes
Most poor answers in retrieval-augmented systems trace back to the retriever rather than the language model. The failures below are frequent in practice, and each has a well-understood remedy.
- Vocabulary mismatch: users and documents describe the same thing differently; query rewriting, query expansion or a dense model helps.
- Identifier blindness: dense retrievers miss product codes and error numbers; adding a sparse component fixes most cases.
- Lost context: chunks lack the surrounding document title or section; contextual retrieval or parent document retrieval restores it.
- Redundant results: the top k passages repeat one another; maximal marginal relevance improves diversity.
- Embedding drift: documents and queries are encoded with different model versions; re-index the whole collection after any model change.
A further warning concerns access control. A retriever that ignores document permissions can surface confidential passages to unauthorized users, so permission filtering belongs inside retrieval, not after generation.
A non-technical view
Think of a retriever as a very fast librarian. You ask a question, and before anyone writes an answer, the librarian runs to the shelves and brings back the few pages most likely to help. The librarian does not write the reply. It only chooses what to read.
If the librarian brings the wrong pages, even a brilliant writer will give a weak answer. That is why good retrieval comes first: an AI assistant can only be as accurate as the material it is handed.
A concrete example
An internal support assistant at a mid-sized software company answers employee questions from about 40,000 policy and product documents. The first version used only a dense retriever returning the top 5 chunks. Questions that mentioned exact error codes, such as a specific license failure number, often returned unrelated passages because the embedding model treated the codes as noise.
The team added a BM25 index, fused both lists with reciprocal rank fusion, raised the candidate count to 100 and inserted a cross-encoder reranker that kept the best 6. On a test set of 300 real employee questions, recall at 100 rose noticeably and the share of answers judged correct by support staff improved, while median retrieval latency stayed under 150 milliseconds. The lesson was that the retriever, not the language model, was the bottleneck.