Question answering is the natural language processing task in which a system receives a question phrased in ordinary language and returns a direct answer rather than a list of documents. The answer may be a short span of text, a generated sentence, a number from a table or a yes-or-no verdict.
In Speech & language AI, question answering sits where natural language understanding meets information access. It combines reading comprehension, retrieval and natural language generation, and it underpins search answer boxes, support assistants, voice assistants and enterprise knowledge tools.
How question answering systems work
A question answering pipeline has three conceptual steps: understand the question, find relevant evidence and produce the answer. Classic systems implemented each step as a separate component, while modern systems often merge them inside a large language model with a retrieval layer.
Question understanding identifies the expected answer type (a person, a date, a quantity, an explanation), key entities and constraints such as time ranges. Evidence retrieval then searches a document collection, a knowledge graph or a database. Finally, the system either extracts a span from the evidence or generates a fluent answer conditioned on it.
The source of knowledge defines the main variants of the task, as the following table shows.
| Variant | Knowledge source | Typical answer form |
|---|---|---|
| Extractive reading comprehension | One given passage | Exact text span |
| Open-domain question answering | Large document collection | Span or generated sentence |
| Knowledge base question answering | Knowledge graph triples | Entity or value |
| Table and database question answering | Tables or SQL databases | Cell, aggregate or query result |
| Closed-book question answering | Model parameters only | Generated text |
| Visual question answering | An image plus text | Short phrase |
Extractive and generative approaches
Extractive models, typically encoder transformers fine-tuned on reading comprehension datasets, predict a start and an end position inside the passage. Their answers are always grounded in the text, but they cannot combine facts from several places or rephrase an answer for clarity.
Generative models write the answer token by token. They handle multi-hop reasoning, summarizing and explaining, yet they can produce hallucinations: confident statements unsupported by any evidence. Pairing a generator with retrieval, known as retrieval-augmented generation, keeps answers tied to current sources and lets the system show which passages it used.
| Property | Extractive | Generative |
|---|---|---|
| Grounding | Answer copied from text | Depends on prompt and retrieval |
| Multi-hop reasoning | Weak | Strong |
| Hallucination risk | Very low | Moderate to high without retrieval |
| Compute cost | Low | Higher, grows with answer length |
| Answer style | Fragment | Complete sentences |
Question answering among neighboring language tasks
Question answering overlaps with several Terms in its Topic, but its goal is the answer itself. A chatbot manages an entire conversation and may answer questions as one of many skills. Intent recognition classifies what a user wants, which is often the first step before a question is routed to a question answering component.
Named entity recognition helps by detecting the people, places and products a question mentions, while text summarization condenses documents without targeting a specific query. In voice assistants, speech-to-text converts the spoken question first, so recognition errors propagate directly into wrong answers.
The task also differs from information retrieval, which ranks documents by relevance and leaves reading to the user. Question answering takes on that reading step, so its errors are more visible: a search engine that ranks a weak page third is tolerable, while an assistant that states a wrong figure as fact can mislead a user who never opens the source. This difference explains why question answering systems need stricter grounding and abstention rules than ordinary search.
Evaluating question answering quality
Evaluation measures whether answers are correct, complete and supported. Extractive benchmarks rely on string overlap, while generative systems increasingly need human judgment or model-based graders because a correct answer can be worded in many ways.
| Measure | What it checks | Limitation |
|---|---|---|
| Exact match | Answer equals the reference string | Penalizes valid paraphrases |
| Token F1 | Word overlap with the reference | Ignores factual errors in extra words |
| Answer faithfulness | Support in retrieved evidence | Needs a reliable judge |
| Retrieval recall at k | Evidence present in top k results | Does not judge the final answer |
| Abstention accuracy | Correct refusal when no answer exists | Rarely reported |
A system that refuses to answer unanswerable questions is often more valuable than one that scores slightly higher on accuracy but invents answers. Test sets should therefore include deliberately unanswerable questions and questions whose answers changed over time.
Designing a reliable question answering system
Most failures in production come from retrieval rather than the reader model. If the right passage never reaches the model, no amount of model quality can recover the answer. Teams therefore invest in chunking, hybrid keyword and vector search, and reranking.
| Symptom | Likely cause | Fix |
|---|---|---|
| Plausible but wrong answers | Weak grounding | Require citations to retrieved passages |
| Outdated facts | Closed-book answering | Retrieve from a maintained source |
| Answer split across documents | Single-hop retrieval | Iterative or multi-query retrieval |
| Correct passage, wrong span | Poor chunk boundaries | Overlapping chunks of 200-500 tokens |
| Confident answers to unknowns | No abstention policy | Confidence threshold and refusal |
- Keep a gold question set drawn from real user queries and rerun it after every change.
- Log questions without answers; they reveal gaps in the underlying content.
- Separate permissions at retrieval time so users never receive answers from documents they cannot access.
A plain-language summary
A search engine gives you a list of pages. A question answering system tries to give you the answer itself. You ask when a contract ends, and it replies with the date.
It works a bit like a good librarian. First it works out what you are really asking. Then it finds the right book and page. Finally it reads the passage and tells you the answer. The most trustworthy systems also show where the answer came from, so you can check it.
An example from practice
An insurance company has about 4,000 policy documents and internal guidelines that call center agents must consult during calls. Agents previously searched a document portal and needed several minutes to find clauses about exclusions or waiting periods.
The company deploys a retrieval-augmented question answering assistant. It splits documents into overlapping chunks of about 400 tokens, retrieves the top eight with hybrid search and asks a language model to answer only from those passages, with clause references. When retrieval confidence is low, the assistant says it cannot find the answer and suggests a specialist queue.
After a pilot, supervisors report that agents find policy answers in well under a minute in most cases, and a weekly review of logged unanswered questions leads to rewriting several unclear guidelines.