Hallucination rate

On this page 8

Hallucination rate is the proportion of model outputs, or of claims within those outputs, that are false, fabricated or unsupported by the provided source or by verifiable facts. In model evaluation for artificial intelligence, it is one of the main metrics used to judge whether a language model can be trusted for factual answers, summaries and retrieval-based tasks.

The figure is always relative to a definition, a dataset and a judging method. A model can show a low hallucination rate on short summaries of supplied documents and a much higher one on open questions about obscure facts, so a number without its test conditions says little.

How hallucination rate is calculated

Hallucination rate is calculated by dividing the number of hallucinated units by the total number of evaluated units. The unit can be a whole response, a sentence or an atomic claim, and that choice changes the result substantially.

Response-level scoring marks an answer as hallucinated if it contains at least one unsupported statement. Claim-level scoring breaks each answer into individual factual statements and checks each one, which gives a finer and usually lower percentage for long answers.

UnitCounted as hallucinated whenEffect on the rate
Response levelAny part of the answer is unsupportedHigher for long answers
Sentence levelThe sentence contains an unsupported claimModerate granularity
Atomic claim levelThe single factual statement is unsupportedFinest, often lower per claim
Citation levelA cited source does not support the claimSpecific to grounded answers
Units of measurement and their effect

Many benchmarks also track the abstention rate, meaning how often the model declines to answer. A model that refuses frequently can show a low hallucination rate while being far less useful, so the two figures should be reported together.

Types of hallucination being measured

Evaluations distinguish intrinsic and extrinsic hallucinations, and the distinction determines what counts as an error. Intrinsic errors contradict the given source, while extrinsic errors add information that the source cannot confirm.

TypeDefinitionTypical task
IntrinsicOutput contradicts the supplied inputSummarization, translation
ExtrinsicOutput adds content not verifiable from the inputSummarization, retrieval answers
FactualOutput conflicts with established world knowledgeOpen-domain question answering
Citation fabricationReferences or quotes that do not existResearch and legal assistance
Instruction driftOutput ignores constraints and invents detailsAgent and tool use
Main hallucination categories

In grounded settings such as retrieval-augmented generation, extrinsic content may be true in the real world but still counts as a hallucination, because the system was required to answer only from the retrieved documents.

Methods for detecting hallucinations

Hallucinations are detected by human annotation, automated judges or reference-based checks, and each method has a different cost and reliability profile. Large evaluations usually combine automated scoring with a human-audited sample.

MethodStrengthWeakness
Expert human reviewHighest accuracy on nuanced claimsSlow, costly, needs agreement checks
LLM-as-a-judgeFast and scalableJudge model can itself err or be biased
Entailment modelsConsistent source-to-claim checksStruggle with paraphrase and numbers
Reference matchingObjective for short factual answersFails on open-ended text
Self-consistency samplingNo reference neededConsistent errors go undetected
Detection methods compared

Whatever the method, reliable studies report inter-annotator agreement or judge-to-human agreement. Without that, a reported hallucination rate may reflect the judge’s errors as much as the model’s.

Factors that change the measured rate

The measured hallucination rate depends heavily on task type, question difficulty, decoding settings and grounding. Comparing figures across different benchmarks is therefore misleading unless the conditions match.

FactorTends to raise the rateTends to lower the rate
Knowledge typeRare entities, recent eventsCommon, well-documented facts
GroundingNo source documents suppliedRelevant retrieved context
Answer lengthLong, detailed outputsShort, constrained outputs
Sampling temperatureHigh temperatureLow or greedy decoding
Abstention policyModel pressured to always answerModel allowed to say it is unsure
Conditions that raise or lower the rate

Training incentives also matter. Benchmarks that reward any answer over no answer encourage confident guessing, while calibration-aware scoring that penalizes wrong answers more than abstentions pushes models toward admitting uncertainty.

Reducing the hallucination rate in practice

Teams reduce the hallucination rate by combining grounding, decoding control, training changes and output verification, because no single technique removes fabrication entirely. The appropriate mix depends on whether the application needs open knowledge or answers drawn from a fixed corpus.

  • Retrieval grounding supplies relevant documents so the model can quote rather than recall.
  • Citation requirements force each claim to point to a passage that a checker can verify.
  • Lower sampling temperature reduces creative but unsupported continuations.
  • Preference training for abstention teaches the model to state uncertainty instead of guessing.
  • Post-generation verification uses a second model or rule set to flag unsupported claims before display.

Each intervention should be validated on the same fixed evaluation set before and after the change. Otherwise, an apparent improvement may come from easier prompts or a different judge rather than from genuinely more faithful outputs.

Limitations and common misreadings

A single hallucination rate is not a universal property of a model. It is a measurement on one distribution of prompts, and the same model can vary by an order of magnitude between a summarization benchmark and an obscure-fact quiz.

  • Benchmark contamination can make a model look more factual than it is on unseen questions.
  • Low rate with high abstention may hide poor coverage rather than better knowledge.
  • Judge drift occurs when the automated judge model changes between evaluations.
  • Severity is ignored by a plain rate: a wrong date and a fabricated legal precedent count the same.

For deployment decisions, teams therefore pair the rate with domain-specific test sets, severity weighting and error analysis of real failures rather than relying on a public leaderboard figure alone.

In plain words

Hallucination rate tells you how often an AI makes things up. If a chatbot answers 100 questions and 7 answers contain invented facts, its hallucination rate on that test is 7 percent.

The number is only meaningful with its context. Easy questions give low rates, hard or obscure questions give high ones. A good result also requires that the model does not simply refuse to answer whenever it is unsure.

A real-world example

A legal services firm tests an AI assistant that summarizes court decisions for its lawyers. The evaluation team prepares 300 decisions with expert-written reference summaries and breaks each model summary into individual factual claims, checked by two paralegals against the source text.

The first version shows about 9 percent of claims unsupported, many of them invented case citations. After the team adds retrieval of the full decision text and instructs the model to quote only from it, the rate falls to under 2 percent of claims, with abstentions rising slightly. The firm approves the tool for internal drafting but keeps mandatory human review for any citation used in a filing.