Knowledge graph question answering

Also known as: KGQA

On this page 8

Knowledge graph question answering (KGQA) is the task of answering a natural language question by locating entities and relations in a knowledge graph and returning the facts, values or entity sets that satisfy the question. Within the Retrieval & knowledge area of artificial intelligence, KGQA is the structured counterpart of text retrieval: instead of ranking passages, it executes a precise path or query over typed triples such as (company, headquartered_in, city).

The appeal of KGQA is exact, verifiable answers. When a question asks for a count, a comparison or a chain of relations, a graph can return a result that is traceable to specific edges, which a language model generating from memory cannot guarantee. The price is coverage: a KGQA system can only answer what the graph actually contains and what its schema can express.

How knowledge graph question answering resolves a question

A KGQA system turns an open question into an operation over the graph through a sequence of well-defined stages. The first stage is entity linking, which maps surface mentions such as “the Rhine” to a unique node identifier. The second is relation detection, which decides which predicates in the schema correspond to phrases such as “flows through” or “was founded by”.

The final stages assemble a logical form or graph query, execute it and format the answer. In classical systems the logical form is a SPARQL or Cypher query; in neural systems it may be an implicit path scored by a model. Errors compound across stages, so entity linking accuracy often caps end-to-end accuracy regardless of how strong the later components are.

The table below lists the main stages, what each produces and the typical failure at that point.

StageOutputTypical failure
Entity linkingNode identifiers for mentioned entitiesAmbiguous names linked to the wrong node
Relation detectionCandidate predicates from the schemaParaphrase not matched to any predicate
Query constructionSPARQL, Cypher or a scored pathMissing constraint or wrong join direction
ExecutionBindings, counts or entity setsEmpty result from incomplete data
Answer generationNatural language replyCorrect bindings phrased misleadingly
Stages of a KGQA pipeline

Main approaches to KGQA

KGQA methods fall into a few families that differ in how explicitly they represent the query. Semantic parsing approaches produce an executable query, which makes results auditable. Retrieval-based approaches rank candidate answer nodes in the neighborhood of the linked entities, which tolerates schema mismatch but hides the reasoning.

Since large language models became common, a third family uses the model as a query generator or graph navigator. The model either writes a query from a schema description, an approach often called text-to-SPARQL or text-to-Cypher, or walks the graph step by step as an agent that calls lookup tools. The comparison below summarizes the trade-offs.

ApproachHow it worksStrengthWeakness
Semantic parsingMaps question to a formal queryExact and auditableBrittle on unseen phrasings
Embedding-based retrievalScores candidate nodes in vector spaceRobust to paraphraseWeak on aggregation and negation
LLM query generationModel writes SPARQL or Cypher from schemaLittle training data neededInvalid or hallucinated predicates
LLM graph explorationAgent expands nodes iterativelyHandles large, messy graphsHigh latency and token cost
Approaches compared

Many production systems are hybrids: an LLM drafts a query, a validator checks it against the schema, and a fallback retriever handles questions the graph cannot express.

Question types and their difficulty

Question complexity is the main driver of KGQA accuracy. Single-hop questions need one triple, while multi-hop questions chain several relations and multiply the chances of error at each step. Questions with aggregation, comparison or temporal constraints require operators that pure retrieval methods do not represent well.

The following table groups common question types by the graph operation they require.

Question typeExampleRequired operation
Single-hopWho directed a given filmOne triple lookup
Multi-hopWhere was the director of that film bornPath of two or more edges
AggregationHow many rivers cross a countryCount over a result set
ComparisonWhich of two towers is tallerNumeric filter or ordering
TemporalWho led the agency in 2015Qualifier or time-interval filter
BooleanIs a city the capital of its stateASK query or existence check
Question types and required graph operations

Public benchmarks such as WebQuestionsSP, ComplexWebQuestions, LC-QuAD and MetaQA are organized largely along these dimensions. Reported accuracy on single-hop sets is usually far higher than on complex sets, and results vary considerably with the graph version used.

KGQA compared with RAG and GraphRAG

KGQA differs from text-based RAG in the unit it retrieves: RAG retrieves passages and lets a model read them, while KGQA retrieves structured facts with explicit relations. GraphRAG sits between the two, building a graph from documents and using community summaries or graph neighborhoods as context for generation rather than executing exact queries.

PropertyKGQAText RAGGraphRAG
Retrieved unitTriples and pathsText chunksEntities, relations and summaries
Answer precisionHigh for covered factsDepends on passage qualityModerate
Aggregation supportNative through queriesPoorPartial
Schema neededYesNoExtracted automatically
Main riskIncomplete graphIrrelevant chunksExtraction noise
KGQA and related retrieval paradigms

The choice is rarely exclusive. A common pattern is query routing: factual, countable questions go to the graph, while explanatory questions go to semantic search over documents, and the answers are combined during grounding.

Designing and evaluating a KGQA system

Practical KGQA design starts with the schema and the entity catalog, not the model. Clear predicate names, aliases for entities and consistent identifiers raise linking accuracy more than a larger language model does. Supplying the model with a compact schema description and a few example queries is usually more effective than exposing the full ontology.

  • Validate generated queries against the schema before execution and reject unknown predicates.
  • Return the executed query or path with the answer so users and auditors can check it.
  • Distinguish empty results from false answers, because a missing edge does not prove a negative.
  • Measure accuracy per stage, including entity linking precision, query exact match and answer F1 or Hits@1.

Evaluation should use answer-level metrics such as Hits@1 or F1 over answer sets together with stage-level diagnostics. A system that scores well on answers but generates invalid queries for a large share of questions is fragile, because its successes may depend on fallback retrieval rather than on the graph.

Limitations and common mistakes

The most serious limitation is the open-world assumption: real graphs are incomplete, so the absence of a triple usually means unknown rather than false. Systems that answer “no” whenever a query returns nothing produce confident errors. Graph staleness causes a related problem, since facts change faster than many graphs are refreshed.

Other frequent mistakes include ignoring qualifiers such as start and end dates, which turns temporal questions into wrong answers, and letting a language model paraphrase results freely, which can reintroduce hallucinated details after an exact retrieval. Finally, benchmark scores rarely transfer directly to enterprise graphs, whose schemas, naming conventions and question styles differ substantially from public datasets.

Explained without jargon

A knowledge graph is a large map of facts. People, products and places are dots. The lines between them carry labels such as “works for” or “located in”. Knowledge graph question answering lets you ask that map a plain question and get back the exact fact.

The system first finds the dots the question mentions. Then it picks the lines that matter and follows them to the answer. Every answer comes from specific connections, so it can be checked. The main catch is that the map must already contain the fact. If it does not, the system cannot invent it and should say that it does not know.

A typical use case

A pharmaceutical research team maintains a knowledge graph of roughly two million nodes linking compounds, protein targets, clinical trials and reported adverse events. Analysts often ask questions such as which approved drugs that inhibit a given enzyme have trials with reported liver toxicity. Answering this from documents takes hours of reading, because the facts are spread across several sources.

The team deploys a KGQA assistant in which a language model generates a Cypher query from a short schema description, a validator rejects queries with unknown labels, and the executed query is shown beside the answer. On an internal test set of 300 analyst questions, most multi-hop questions were answered correctly with a visible query path, and failures were traced mainly to entity linking of drug synonyms. Adding an alias table for brand and generic names was the single change that raised accuracy most, which confirmed that the catalog mattered more than the model.