Hybrid search is a retrieval approach that combines lexical keyword search with semantic vector search and merges their results into one ranked list. Lexical retrieval, usually BM25, matches exact terms; semantic retrieval compares embeddings to match meaning. In retrieval & knowledge systems for artificial intelligence, hybrid search has become the default first stage of retrieval-augmented generation (RAG) because each method covers the other’s blind spots.
The core insight is that the two signals fail in different ways. Keyword search misses paraphrases and synonyms, while vector search misses rare identifiers, product codes, names and exact phrases. Combining them raises recall across a wider variety of queries than either method achieves alone.
How hybrid search works
A hybrid search system runs two retrievers in parallel over the same corpus and then fuses their outputs. A typical pipeline has four stages, and each one has its own tuning choices.
- Indexing: each document chunk is stored both in an inverted index for keyword search and as a dense vector in an approximate nearest neighbor index such as HNSW.
- Parallel retrieval: the query is tokenized for BM25 and embedded for vector similarity, and each retriever returns its own top candidates, commonly 20 to 100.
- Fusion: the two candidate lists are merged into one ranking by rank-based or score-based fusion.
- Optional reranking: a cross-encoder reranker rescores the top fused results, often the best 20 to 50, before they are passed to a language model.
The fusion step is where hybrid search is decided. Because BM25 scores and cosine similarities live on different scales, they cannot simply be added; they must be converted to ranks or normalized first.
Fusion methods and key parameters
The two main fusion families are rank-based fusion, which ignores raw scores, and score-based fusion, which normalizes and weights them. The table compares the methods used most often.
| Method | How it combines results | Strength | Weakness |
|---|---|---|---|
| Reciprocal rank fusion | Sums 1 / (k + rank) across lists, k often 60 | Robust, no score calibration needed | Ignores how confident each retriever is |
| Weighted linear combination | Normalizes scores, then alpha × vector + (1 − alpha) × keyword | Tunable balance | Sensitive to normalization choice |
| Distribution-based normalization | Scales scores by mean and standard deviation | Handles score outliers better | Needs stable score distributions |
| Learned fusion | A model weights features from both retrievers | Best quality with training data | Requires labeled relevance data |
Reciprocal rank fusion is popular because it needs no calibration at all. A document ranked first by one retriever and tenth by the other still scores well, while a document found by only one retriever is not discarded. The constant k dampens the advantage of top positions, so agreement between both retrievers becomes the strongest signal. Its drawback is that it treats a narrow win and a decisive win identically, which a tuned weighted fusion can exploit.
The most important tuning knob in weighted fusion is the alpha weight. Values around 0.5 to 0.7 toward semantic search suit conversational questions, while catalogs dominated by part numbers or legal citations often benefit from more keyword weight. Alpha should be chosen on a labeled evaluation set of real queries, not by intuition.
Hybrid search compared with single-method retrieval
Hybrid search sits between pure lexical search and pure semantic search, and its value is clearest when queries are mixed. The table shows how each approach behaves on typical query types.
| Query type | Keyword search (BM25) | Semantic search | Hybrid search |
|---|---|---|---|
| Exact code or identifier | Strong | Weak, codes embed poorly | Strong |
| Paraphrased question | Weak on vocabulary mismatch | Strong | Strong |
| Rare proper name | Strong | Often unreliable | Strong |
| Conceptual or vague query | Weak | Strong | Strong |
| Multilingual query | Weak across languages | Strong with multilingual embeddings | Strong |
Hybrid search is different from reranking, which reorders an existing candidate list, and from query rewriting, which changes the query before retrieval. In mature systems all three work together: rewriting improves the query, hybrid retrieval maximizes recall, and reranking maximizes precision at the top.
Implementing hybrid search in practice
Most modern search engines and vector databases now offer built-in hybrid queries, so the practical work lies mainly in configuration and evaluation. Some systems also support learned sparse vectors, which add term expansion to keyword matching and can replace classic BM25 in the lexical branch.
| Decision | Common default | When to change it |
|---|---|---|
| Candidates per retriever | 50 to 100 | Increase for large, diverse corpora |
| Fusion method | Reciprocal rank fusion | Switch to weighted fusion when tuned on labels |
| Chunk size | 200 to 500 tokens | Smaller for precise facts, larger for context |
| Reranker | Cross-encoder on top 20 to 50 | Skip only when latency is critical |
| Evaluation metric | Recall at 10 and nDCG at 10 | Add answer quality when used for RAG |
The lexical branch needs the same text preprocessing care as classic search: language-specific analyzers, stemming and stop-word handling still matter. Chunking must keep both branches aligned, so each chunk has one identifier used by both indexes.
Limitations and common mistakes
Hybrid search improves robustness but adds cost and complexity: two indexes to maintain, two queries per request and a fusion step to tune. The following mistakes are common.
- Adding raw scores from BM25 and cosine similarity without normalization, which lets one retriever dominate silently.
- Tuning alpha on a handful of queries instead of a representative evaluation set.
- Retrieving too few candidates from each branch, so fusion has nothing useful to merge.
- Skipping metadata filters such as date or access rights, which must apply equally to both branches.
- Assuming hybrid search fixes poor chunking or weak embeddings, which it cannot.
Stripped to the essentials
Imagine asking two librarians for help. One finds books containing your exact words; the other understands what you mean even if you use different words. Each finds things the other misses.
Hybrid search asks both librarians at once and combines their lists into one, putting the books that both recommend near the top. The result is a search that handles both precise codes and loosely worded questions.
Where it is applied
A manufacturer of industrial pumps builds an assistant for its service technicians on top of about 30,000 pages of manuals and service bulletins. In early tests, pure vector search answers questions such as why a pump vibrates at low flow, but it fails on queries like “seal kit SK-4471 torque values”, returning unrelated seal procedures.
The team adds a BM25 index over the same chunks, merges both lists with reciprocal rank fusion and reranks the top 30 results. On an evaluation set of 400 real technician questions, the share of queries whose correct passage appears in the top five rises noticeably, with the largest gains on part numbers and error codes. Technicians now get correct procedures for both conceptual and code-based questions from one search box.