Query expansion

On this page 7

Query expansion is the technique of adding related terms, phrases or generated text to a user query before retrieval, so that the search system matches relevant documents even when they use different words than the user. In the Retrieval & knowledge part of artificial intelligence, it addresses the vocabulary mismatch problem: a person asks about “heart attack” while the best document says “myocardial infarction”.

The method is decades old in classical information retrieval, but it has gained new importance in retrieval-augmented generation (RAG), where a missed passage means the language model answers without the evidence it needed. Modern variants use large language models to generate expansions, while older variants rely on thesauri, ontologies and statistics from the corpus itself.

How query expansion improves recall

Query expansion works by enlarging the set of signals the retriever can match against the index. A short query such as “laptop overheating” contains only two content words, so a lexical engine like BM25 can only score documents that contain those exact tokens. Adding “thermal throttling”, “fan noise” and “CPU temperature” gives the engine more chances to hit relevant text.

The primary goal is higher recall, meaning that more of the relevant documents appear in the candidate set. The cost is a potential loss of precision, because every added term can also match irrelevant documents. This is why expansion terms are usually down-weighted relative to the original terms, often with a weight between 0.3 and 0.7, rather than treated as equal parts of the query.

A typical pipeline has four stages, and each stage offers a design decision that affects quality and latency.

StageWhat happensMain design choice
Candidate generationRelated terms or texts are proposedThesaurus, corpus statistics or LLM
FilteringOff-topic or ambiguous candidates are removedSimilarity threshold or domain list
WeightingEach added term receives an importance scoreFixed weight or learned weight
Query reformulationOriginal and added terms are combinedBoolean OR, weighted sum or multiple queries
Stages of a query expansion pipeline

Main families of query expansion methods

Query expansion methods differ mainly in where the extra terms come from. Global methods draw on resources that exist before the query arrives, while local methods analyze the documents retrieved for that specific query. Generative methods, the newest family, ask a language model to write the expansion.

Pseudo-relevance feedback (PRF) is the best-known local method. The system runs the query once, assumes the top 5 to 20 documents are relevant, extracts their most distinctive terms and runs an expanded second query. The Rocchio algorithm and the RM3 relevance model are classic implementations, and both remain strong baselines on lexical indexes.

The following table compares the common families by their source of terms and their typical strengths.

Method familySource of added termsStrengthTypical weakness
Thesaurus-basedWordNet, domain vocabularies, ontologiesPredictable and explainablePoor coverage of new jargon
Pseudo-relevance feedbackTop documents from a first retrieval passAdapts to the corpusDrifts when first results are poor
Embedding neighborsNearest words in an embedding spaceCaptures informal synonymsAdds related but wrong terms
LLM-generated expansionText written by a language modelRich, context-aware wordingLatency, cost and hallucinated facts
Learned sparse expansionModel-predicted vocabulary weightsBuilt into the indexRequires a trained encoder
Families of query expansion methods

Learned sparse models such as SPLADE perform a form of document-side and query-side expansion inside the encoder, predicting weights for vocabulary terms that never appeared in the text. In such systems, an explicit expansion step can add less value than it does for plain BM25.

Query expansion is one of several query-side techniques that improve retrieval, and the boundaries between them are not always sharp in practice. The essential distinction is that expansion keeps the original query and adds to it, while other techniques replace, split or route it.

TechniqueWhat it changesRelation to query expansion
Query rewritingReplaces the query with a clearer versionSubstitutes rather than adds
HyDEGenerates a hypothetical answer document and embeds itA generative expansion for dense retrieval
Multi-query retrievalRuns several paraphrases and merges resultsExpansion spread across parallel queries
Query routingSends the query to the right index or toolChooses a target, adds no terms
RerankingReorders candidates after retrievalRestores precision lost to expansion
How query expansion differs from neighboring techniques

In dense retrieval, adding words to the query text shifts the query embedding toward a broader region of the vector space. That effect can help a vague query but can blur a precise one, so dense systems often prefer multi-query retrieval with reciprocal rank fusion over concatenating many terms into one query.

Applying query expansion in RAG systems

In a RAG pipeline, query expansion is most useful when user questions are short, informal or domain-ambiguous and the knowledge base uses formal terminology. A common production pattern asks an LLM to produce three to five alternative phrasings or a short list of domain terms, retrieves for each, fuses the ranked lists and passes the top candidates to a cross-encoder reranker.

The decision to expand should be driven by evaluation, not by habit. Teams measure recall at k, mean reciprocal rank and answer faithfulness on a labeled query set with and without expansion, because the effect varies strongly between corpora.

SituationRecommended approachReason
Short keyword queries on BM25Pseudo-relevance feedback (RM3)Cheap and consistently helpful
Domain jargon in documentsCurated synonym or ontology listControlled, auditable mappings
Conversational RAG questionsLLM multi-query with fusionHandles paraphrase and intent
Exact identifiers or error codesNo expansion or exact-match boostAdded terms dilute the signal
Strict latency budget below 100 msOffline or index-side expansionAvoids an extra model call
When to apply query expansion

Limitations and common mistakes

The central risk of query expansion is query drift: the added terms pull retrieval toward a topic the user did not intend. An expansion of “Java” with “coffee” and “Indonesia” ruins a programming query, and pseudo-relevance feedback amplifies the error when the first-pass results are already off-topic.

  • Equal weighting of added terms lets weak expansions outvote the original intent.
  • Expanding precise queries such as product codes, legal citations or names usually lowers precision.
  • Unchecked LLM expansions may introduce invented facts that steer retrieval toward wrong passages.
  • Ignoring latency is costly, because a generative expansion adds a full model call of roughly 200 ms to several seconds.
  • Skipping a reranker leaves the precision loss from expansion uncorrected.

Expansion also complicates debugging and explainability, because the documents returned no longer match the words the user typed. Logging the expanded query alongside the original is a simple practice that makes retrieval failures much easier to diagnose.

A non-technical view

Imagine asking a librarian for books about “cars”. A good librarian also checks the shelves for “automobiles”, “vehicles” and “motoring”, because the best book may not use your exact word. Query expansion is that librarian habit built into software.

The search system quietly adds related words or alternative questions to what you typed, so it finds useful material that uses different vocabulary. The trick is balance: a few good extra words help, while too many lead the search off track.

A concrete example

An internal IT support assistant at a mid-sized company answers employee questions from about 8,000 help articles. Employees type queries such as “VPN keeps dropping at home”, while the articles use phrases like “tunnel disconnect”, “split tunneling” and “MTU configuration”. In early tests, the correct article appeared in the top five results for only about six in ten questions.

The team adds an expansion step: a small language model writes three alternative phrasings using IT terminology, each phrasing is searched with hybrid BM25 and vector retrieval, and the lists are merged with reciprocal rank fusion before a cross-encoder reranks the top 30. Exact error codes bypass expansion entirely. On a labeled set of 400 questions, recall at five rises to roughly eight in ten, and the assistant cites the correct article far more often, at the cost of about 300 ms of extra latency.