Open-domain question answering

Also known as: open-domain QA

On this page 7

Open-domain question answering (open-domain QA) is the task of answering natural-language questions about any topic by first finding relevant evidence in a large, unrestricted collection of documents and then producing an answer from that evidence. In artificial intelligence, it sits at the center of retrieval and knowledge work, because the system receives no passage in advance and must locate the facts itself.

The contrast with closed-domain or reading-comprehension QA is decisive: there, the relevant paragraph is supplied with the question. Open-domain systems must instead search millions of passages, which makes retrieval quality the main bottleneck and turns the problem into a combination of information retrieval and machine reading.

How open-domain question answering works

Most open-domain QA systems follow a retriever-reader architecture, in which a fast search component narrows a huge corpus to a handful of candidate passages and a slower, more accurate model reads them to extract or generate the answer. The design separates recall, which the retriever must secure, from precision, which the reader must deliver.

The retriever can be sparse, scoring term overlap with methods such as BM25 or TF-IDF, or dense, comparing learned embeddings of the question and the passages through vector similarity. Many production systems add a reranking stage with a cross-encoder that scores each question-passage pair jointly before the reader sees it.

The following table lists the typical stages of a pipeline and what each one contributes to the final answer.

StageMain functionTypical methodTypical output size
IndexingSplits and encodes the corpusChunking, embedding, inverted indexMillions of passages
RetrievalFinds candidate evidenceBM25, dense bi-encoder, hybrid search20 to 100 passages
RerankingOrders candidates by relevanceCross-encoder or late interaction3 to 20 passages
ReadingProduces the answerSpan extractor or generative modelOne short answer
VerificationChecks support and confidenceAnswer scoring, citation checksAnswer or abstention
Stages of a typical open-domain QA pipeline

Extractive, generative and closed-book approaches

Open-domain QA systems differ mainly in how the reader forms the answer and whether retrieval happens at all. An extractive reader selects an exact text span from a retrieved passage, while a generative reader writes the answer in its own words, usually by conditioning a language model on several passages at once.

A third variant, closed-book QA, skips retrieval and asks a large language model to answer from its parametric knowledge alone. It is simple to deploy but cannot cite evidence, cannot learn facts that changed after training, and is more prone to confident fabrication. The table compares the three approaches.

ApproachAnswer formMain strengthMain weakness
Extractive retriever-readerExact span from a passageVerifiable, low hallucinationCannot combine facts across passages
Generative retriever-readerFree-text answerFuses evidence from many passagesMay add unsupported details
Closed-book modelFree-text answerNo index to maintainOutdated or invented facts
Multi-hop retrieval systemAnswer built from chained evidenceHandles compositional questionsErrors compound across hops
Comparison of answering approaches

Retrieval-augmented generation, or RAG, is in practice the generative retriever-reader pattern applied beyond benchmarks, so open-domain QA can be seen as the research task from which RAG grew.

Benchmarks and evaluation metrics

Open-domain QA is evaluated with separate metrics for retrieval and for answering, because a correct answer cannot be produced from passages that do not contain it. Retrieval is usually measured by top-k recall, the share of questions for which at least one of the first k passages contains the answer, and by ranking measures such as mean reciprocal rank.

Answer quality is commonly reported as exact match and token-level F1 against reference answers. Widely used datasets include Natural Questions, built from real search queries, TriviaQA, built from trivia questions, and HotpotQA, which requires multi-hop reasoning. Each has known limitations, so results should be read with the dataset’s construction in mind.

MetricWhat it measuresTypical useKnown limitation
Top-k recallAnswer present in first k passagesRetriever evaluationIgnores passage usefulness
Mean reciprocal rankPosition of first relevant passageRanking qualityCounts only one relevant hit
Exact matchAnswer string equals referenceShort factual answersPenalizes valid paraphrases
Token-level F1Word overlap with referencePartial credit for answersRewards overlap, not correctness
Model-based judgingSemantic correctness and supportGenerative answersJudge bias and inconsistency
Common evaluation metrics

Designing a reliable open-domain QA system

Reliable open-domain QA depends more on corpus preparation and retrieval design than on the choice of reader model. Passage length, overlap and metadata decide whether an answer survives chunking intact, and a passage split in the wrong place can make a fact unretrievable.

The decision table below summarizes common situations and the design choices that usually serve them best.

SituationRecommended choiceReason
Many rare names and codesHybrid sparse and dense retrievalSparse search matches exact identifiers
Paraphrased user questionsDense retrieval with query rewritingEmbeddings capture meaning, not wording
Answers spread across documentsMulti-hop or iterative retrievalOne query rarely finds all evidence
High cost of wrong answersAnswer abstention thresholdDeclining is safer than guessing
Frequently changing factsIncremental reindexingStale indexes return outdated evidence
Design decisions for common situations

A practical rule is to measure retrieval recall first. If the answer appears in the top 20 passages for less than about 80 percent of test questions, improving the reader will rarely help, and effort belongs in the index and the retriever.

Limitations and common failure modes

Open-domain QA systems fail in characteristic ways, and most failures trace back to missing, ambiguous or conflicting evidence rather than to weak language understanding.

  • Retrieval misses: the answer exists in the corpus, but the retriever ranks it too low for the reader to see it.
  • Ambiguous questions: a question such as one about a common surname has several valid answers, and the system silently picks one.
  • Conflicting sources: outdated and current passages disagree, and the reader may favor the more frequent but wrong one.
  • Unsupported generation: a generative reader adds plausible details that no retrieved passage contains, which weakens grounding.
  • Benchmark overfitting: high exact-match scores on public datasets do not guarantee accuracy on a private corpus with different language.

Mitigation combines citation of supporting passages, confidence-based abstention, freshness metadata and evaluation on questions drawn from the real target domain rather than only from public benchmarks.

The simple explanation

Open-domain question answering is like asking a skilled librarian a question about anything at all. The librarian does not know every fact by heart. Instead, the librarian quickly finds the right books, opens them to the right pages and reads out the answer.

A computer system does the same in two steps. First it searches a huge collection of texts for the few passages most likely to help. Then it reads those passages and writes a short answer. If the search step brings back the wrong pages, even the best reader cannot answer correctly, which is why good searching matters most.

How it is used in practice

A national public health agency maintains about 40,000 guidance documents, press releases and technical reports collected over two decades. Staff at its information line receive questions such as which vaccine schedule applies to adults who travel abroad, and searching the archive by keyword often returns hundreds of loosely related files.

The agency builds an open-domain QA service over the archive. Documents are split into passages of roughly 300 words with publication dates as metadata, a hybrid retriever returns the top 50 passages, a cross-encoder reranker keeps the best 5, and a generative model writes an answer that cites each supporting passage. Questions whose best passage scores below a set threshold are routed to a human expert.

During a three-month pilot, the team checks a sample of answers every week. Retrieval recall in the top 50 reaches about 90 percent, answers that cite superseded guidance fall after a date filter is added, and the information line handles routine questions noticeably faster while experts concentrate on the unusual cases.