HyDE

Also known as: Hypothetical document embeddings

On this page 7

HyDE, short for hypothetical document embeddings, is a retrieval technique in which a language model first writes a plausible answer to the user query and a dense retriever then searches with the embedding of that generated text instead of the embedding of the query itself. In the field of retrieval & knowledge within artificial intelligence, HyDE belongs to the family of query transformation methods, alongside query rewriting and query expansion.

The idea addresses a basic asymmetry: questions are short and phrased differently from the passages that answer them. By turning a question into something that looks like an answer, HyDE moves the search vector closer to the region of embedding space where relevant documents live, often without any relevance-labeled training data.

How HyDE turns a query into a search vector

HyDE works in two stages: generation and encoding. A generative model receives the query with an instruction such as “write a passage that answers this question”, produces one or more short documents, and an embedding model maps those documents into the same vector space as the indexed corpus.

The generated passage does not need to be factually correct. Its job is to carry the vocabulary, structure and topical signals of a real answer. The encoder acts as a lossy filter: it compresses the text into a dense vector that keeps the broad semantic pattern while discarding many invented specifics. Retrieval then returns real documents from the corpus, so hallucinated details in the draft never reach the final context directly.

The following table summarizes the pipeline as it is typically implemented in a retrieval-augmented generation (RAG) system.

StepComponentOutput
1. PromptingInstruction-following language modelOne to several hypothetical passages
2. EncodingEmbedding model (bi-encoder)One dense vector per passage
3. AggregationMean of passage vectors, optionally with the query vectorA single search vector
4. SearchVector index with approximate nearest neighbor searchTop-k real documents
5. Optional rerankCross-encoder or other rerankerReordered final candidates
Steps of a HyDE retrieval pipeline

In the original formulation, several drafts are sampled and their vectors are averaged together with the query vector. Averaging reduces the influence of any single odd generation and keeps some of the user’s literal intent in the search vector.

Why answer-shaped vectors retrieve better

Dense retrieval relies on vector similarity, usually cosine similarity, between the query and each document. When an embedding model was trained without question-to-passage pairs, short queries and long passages land in different parts of the embedding space, a gap often described as query-document asymmetry. HyDE bridges this gap by comparing document to document rather than question to document.

The benefit is largest when the encoder is unsupervised or out of domain, because such encoders capture document-to-document similarity well but match questions poorly. The original research paired an instruction-tuned generator with an unsupervised contrastive encoder and reported strong zero-shot results on web search and low-resource benchmarks. With modern encoders that are already trained on large question-passage collections, the gain is smaller and less predictable, and in some domains it disappears.

ConditionExpected effectReason
Unsupervised encoderUsually clear improvementEncoder matches documents better than questions
Encoder fine-tuned on queriesSmall or no improvementQuery-document gap already learned
Terse or vague queriesOften helpfulDraft adds missing context and terms
Specialized jargon domainMixed resultsGenerator may lack domain vocabulary
Exact identifiers or codesOften harmfulDraft dilutes the literal token
Conditions that influence the value of HyDE

HyDE is one of several ways to reshape the query before retrieval, and it is easy to confuse with its neighbors. The key distinction is that HyDE produces an answer-shaped document, while query rewriting produces a better question and query expansion adds terms to the existing question.

MethodWhat the model generatesTypical retrieverMain strength
HyDEA hypothetical answer passageDenseCloses the question-answer gap
Query rewritingA clearer reformulated questionDense or sparseFixes ambiguity and conversation references
Query expansionExtra keywords or synonymsMostly sparse (BM25)Improves lexical recall
Multi-query retrievalSeveral alternative questionsDense or sparseCovers different phrasings
Contextual retrievalContext added to indexed chunksDense and sparseImproves the document side instead
HyDE and neighboring retrieval methods

HyDE also differs from reranking: a reranker improves the order of already retrieved candidates, whereas HyDE changes which candidates are retrieved in the first place. The two combine well, because HyDE widens recall and a cross-encoder then restores precision.

Implementing HyDE in a retrieval system

A practical HyDE setup needs only components that most RAG stacks already contain, but several design choices determine whether it helps. The prompt should ask for a passage in the style of the corpus, such as a support article, a contract clause or an abstract, because style alignment matters more than factual accuracy.

  • Number of drafts: one draft is cheapest; three to five drafts reduce variance at a proportional generation cost.
  • Draft length: a short paragraph is usually enough; long drafts drift away from the query focus.
  • Temperature: moderate sampling diversity helps when several drafts are averaged.
  • Query vector inclusion: averaging the original query vector into the search vector protects literal intent.
  • Hybrid search: running BM25 on the original query in parallel and merging with reciprocal rank fusion keeps exact matches.

Teams should evaluate HyDE against a plain dense baseline on their own queries with metrics such as recall at k and mean reciprocal rank. Enabling it globally without measurement is a common mistake, and many production systems apply it selectively through query routing, for example only to short or exploratory questions.

Limitations, costs and failure modes of HyDE

The main cost of HyDE is an extra language model call before every search, which adds latency in the range of hundreds of milliseconds to several seconds depending on model size and draft count. It also adds token cost and makes retrieval less deterministic, since sampled drafts vary between runs.

SymptomLikely causeRemedy
Results follow a wrong assumptionDraft confidently answers a misread questionAverage with the query vector or use several drafts
Exact product codes are missedLiteral identifiers diluted in the draftAdd sparse retrieval and fuse results
Retrieval feels slowGeneration precedes every searchUse a small model, cache drafts, route selectively
No gain over baselineEncoder already handles questions wellDisable HyDE for that index
Bias toward popular viewsGenerator reproduces common beliefsDiversify drafts and review evaluation sets
Common HyDE problems and remedies

A subtler risk is knowledge bias: when the generator holds a wrong belief, the draft steers retrieval toward documents that confirm it. HyDE is therefore weakest on recent events, private data and contested topics, where the generator knows least, which are often exactly the cases for which a RAG system was built.

A non-technical view

Imagine searching a large library by describing the book you want. A short question such as “why do bridges sway” may not resemble any page in the library. HyDE first asks an AI to write a rough sample answer, then looks for real pages that resemble that sample.

The sample answer may contain mistakes, and that is acceptable, because it is only used as a search pattern and is never shown to the user. The key takeaway is simple: searching with a draft answer often beats searching with the bare question, at the cost of a little extra time.

A concrete example

An internal help desk at a mid-sized software company indexes about 40,000 support articles with an off-the-shelf embedding model. Employees type short questions such as “VPN drops after sleep”, and the plain dense search often returns general networking articles instead of the specific troubleshooting page.

The team adds HyDE for queries shorter than eight words. A small language model writes a three-sentence draft in the style of a support article, mentioning power management, adapter settings and reconnection behavior. Its vector is averaged with the query vector, and a BM25 search on the original text is fused with reciprocal rank fusion.

On a test set of several hundred labeled questions, recall at 10 improves noticeably for short queries, while longer, detailed queries show no change, so the routing rule stays. The added latency of roughly half a second is accepted because the correct article now appears among the first results far more often.