Dense retrieval

On this page 7

Dense retrieval is a search method that encodes queries and documents as dense, low-dimensional vectors produced by a neural encoder and ranks documents by vector similarity to the query. In the Retrieval & knowledge area of artificial intelligence, it is the standard first stage of retrieval-augmented generation (RAG), semantic search and open-domain question answering.

The word “dense” contrasts with sparse retrieval, where a document is represented by a vocabulary-sized vector that is mostly zeros. A dense vector typically has 384 to 4,096 dimensions, every one of them carrying a learned value, so two texts can match without sharing a single word when they express the same meaning.

How dense retrieval encodes and matches text

Dense retrieval works in two phases: offline indexing and online querying. During indexing, an embedding model converts each chunk of the corpus into a vector and stores it in a vector index. At query time, the same or a paired encoder turns the question into a vector, and the system returns the chunks whose vectors lie closest to it.

Most systems use a bi-encoder architecture: the query and the document are encoded independently, which means document vectors can be precomputed once and reused for every query. Closeness is measured with cosine similarity, the dot product or Euclidean distance. When vectors are normalized to unit length, cosine similarity and the dot product produce identical rankings.

The encoder learns what “relevant” means through contrastive training. It sees a query, a passage known to answer it and several passages that do not, and it adjusts its weights to pull the matching pair together and push the others apart. Hard negatives, passages that look relevant but are not, teach the model the finest distinctions and strongly influence final quality.

The table below summarizes the components of a typical dense retrieval pipeline and what each one controls.

ComponentRoleMain design choice
ChunkerSplits documents into passagesChunk size and overlap
Document encoderMaps passages to vectorsModel family and dimension
Query encoderMaps questions to vectorsShared or separate weights
Vector indexStores and searches vectorsExact or approximate search
Similarity functionScores query-document pairsCosine, dot product or L2
Components of a dense retrieval pipeline

Key parameters that shape retrieval quality

Retrieval quality depends less on any single setting than on the match between the encoder and the corpus. A general-purpose model trained on web question-answer pairs may perform poorly on legal contracts, source code or clinical notes, where vocabulary and relevance criteria differ sharply.

The following parameters have the largest practical effect, with typical values seen in production systems.

ParameterTypical rangeWhat it affects
Embedding dimension384 to 4,096Expressiveness, memory and latency
Chunk length100 to 1,000 tokensPrecision versus available context
Top-k candidates5 to 100Recall versus downstream cost
Encoder input limit512 to 8,192 tokensLongest passage encoded faithfully
Index typeFlat, HNSW, IVF, DiskANNSpeed versus exact recall
Parameters with the largest effect on results

Larger dimensions are not automatically better. Techniques such as Matryoshka embeddings let a system truncate vectors to a shorter prefix with modest quality loss, and product quantization or binary embeddings compress vectors further. These choices trade a small drop in recall for large savings in memory and cost.

At scale, exact search over every vector becomes too slow, so most deployments rely on approximate nearest neighbor search. An HNSW graph or an IVF index can return results in milliseconds over hundreds of millions of vectors, but it may miss some true nearest neighbors, and that loss should be measured rather than assumed.

Dense retrieval compared with sparse and hybrid approaches

Dense retrieval and sparse retrieval fail in complementary ways. Dense models capture paraphrase and synonymy, while lexical methods such as BM25 and TF-IDF excel at exact identifiers, rare terms, product codes and names the encoder never saw during training.

MethodRepresentationStrengthWeakness
Dense retrievalLearned dense vectorSemantic and paraphrase matchingWeak on rare exact terms
BM25Term frequency statisticsExact keywords, no trainingMisses synonyms
SPLADELearned sparse weightsExpansion with inverted indexLarger index, tuning effort
ColBERTOne vector per tokenFine-grained late interactionMuch larger storage
Hybrid searchDense plus sparse scoresRobust across query typesTwo indexes to maintain
How dense retrieval differs from neighboring methods

In practice, many teams run hybrid search, combining dense and BM25 results with reciprocal rank fusion, and then apply a cross-encoder reranker to the top candidates. The dense stage provides high recall cheaply, and the reranker supplies precision that a bi-encoder alone cannot reach, because it reads query and passage together.

Building and tuning a dense retriever in practice

A reliable dense retriever starts with an evaluation set drawn from real queries, not with a model choice. Without labeled query-passage pairs, teams cannot tell whether a new encoder, chunk size or index setting helps or hurts. Metrics such as recall at k and mean reciprocal rank make those comparisons concrete.

The decision table below maps common situations to a sensible starting configuration.

SituationRecommended choiceReason
General documentation, natural questionsOff-the-shelf embedding modelStrong zero-shot semantic matching
Many part numbers or identifiersHybrid search with BM25Exact tokens need lexical matching
Specialized domain vocabularyFine-tune with hard negativesGeneric models misjudge relevance
Over 100 million passagesQuantized vectors with ANN indexMemory and latency limits
High-stakes answersAdd a cross-encoder rerankerPrecision at the top matters most
Choosing a configuration by situation

Two rules prevent most silent failures. First, queries and documents must be encoded by the same model version, because vectors from different models occupy incompatible spaces. Second, changing the encoder requires re-embedding the entire corpus, which should be planned as a migration rather than a quick configuration change.

Limitations and common failure modes

Dense retrieval is powerful but not a semantic oracle. Its errors are often plausible-looking, which makes them harder to notice than the obvious misses of keyword search.

  • Domain shift: an encoder trained on web text can rank fluent but irrelevant passages above correct technical ones.
  • Negation and numbers: “supports TLS 1.2” and “does not support TLS 1.2” may receive nearly identical vectors.
  • Long documents: a single vector for a long chunk averages many topics and dilutes the signal.
  • Out-of-vocabulary terms: new product names or codes are split into fragments the model represents poorly.
  • Opaque scores: similarity values are relative, so a fixed score threshold rarely transfers between models or corpora.

Mitigations include contextual retrieval, which prepends document context to each chunk before embedding, metadata filtering to restrict the search space, query rewriting to clarify vague questions, and continuous evaluation whenever the corpus or the model changes.

In plain words

Dense retrieval is a way of searching by meaning instead of exact words. A computer turns every piece of text into a list of numbers that acts like a location on a map of ideas. Texts about similar things end up close together on that map.

When someone asks a question, the question also gets a location, and the system simply returns the texts that sit nearest to it. That is why a search for “how to reset my login” can find a page titled “Recovering your account password”. The key point is that closeness on the map stands for closeness in meaning, although the map is not perfect for rare names and codes.

A real-world example

A mid-sized software company runs an internal support assistant over about 40,000 help articles and resolved tickets. Its original keyword search failed whenever customers described a problem in their own words, for example “the app freezes after I sign in” instead of the documented phrase “session initialization timeout”.

The team split the content into chunks of roughly 300 tokens, embedded them with a 768-dimensional model and stored them in an HNSW index. Because ticket queries often contained error codes, they combined the dense results with BM25 through reciprocal rank fusion and reranked the top 50 candidates with a cross-encoder.

On a test set of 500 real customer questions, the share of questions with a correct article in the top five results rose from about 55 percent to about 82 percent. The remaining misses were mostly new error codes, which the team addressed by adding those codes to article metadata.