RAG evaluation

On this page 8

RAG evaluation is the systematic measurement of how well a retrieval-augmented generation system finds relevant evidence and turns that evidence into faithful, correct and useful answers. In the retrieval & knowledge branch of artificial intelligence, it treats the pipeline as two coupled subsystems, the retriever and the generator, and scores each one separately as well as end to end.

The discipline exists because a fluent answer can still be wrong in two different ways: the system may have retrieved the wrong passages, or it may have ignored or distorted the right ones. Without component-level metrics, teams cannot tell which part to fix, and changes to chunking, embeddings or prompts become guesswork.

How RAG evaluation separates retrieval from generation

RAG evaluation works by comparing system outputs against reference data at each stage of the pipeline. A typical evaluation record contains the user question, the retrieved passages, the generated answer and, where available, a gold answer and a list of relevant document identifiers labeled by humans.

Retrieval quality is judged by whether the right passages appear in the top results, which is classic information retrieval measurement. Generation quality is judged by whether the answer is supported by those passages, addresses the question and agrees with the gold answer. End-to-end scores then show the combined effect that a real user experiences.

The following table summarizes the three evaluation layers and the question each one answers.

LayerQuestion answeredTypical inputs
RetrievalDid the system fetch the relevant evidence?Query, ranked passages, relevance labels
GenerationDid the model use the evidence correctly?Query, retrieved context, answer
End to endDid the user get a correct, helpful answer?Query, answer, gold answer or rubric
Layers of RAG evaluation

Core metrics for retrieval and answer quality

RAG evaluation relies on a small set of well-defined metrics, each sensitive to a different failure. Retrieval metrics are usually computed at a cutoff k, such as the top 5 or top 10 passages passed to the model, because only those passages influence the answer.

The table below lists the most widely used metrics, what they measure and their typical range.

MetricWhat it measuresRange and reading
Recall at kShare of relevant passages found in the top k0 to 1, higher is better
Precision at kShare of top-k passages that are relevant0 to 1, higher means less noise
Mean reciprocal rankPosition of the first relevant passage0 to 1, rewards early hits
nDCG at kGraded relevance discounted by rank0 to 1, handles partial relevance
FaithfulnessShare of answer claims supported by context0 to 1, low values signal hallucination
Answer relevanceHow directly the answer addresses the queryScore or rubric grade
Answer correctnessAgreement with a gold answerExact match, F1 or judged score
Common RAG evaluation metrics

Context recall deserves special attention: if the evidence needed for the answer never reaches the model, no prompt engineering can recover it. Many teams therefore treat retrieval recall as the ceiling on end-to-end accuracy and optimize it first.

Human labels versus LLM-as-a-judge scoring

RAG evaluation can be scored by people, by deterministic formulas or by a language model acting as a judge. Each approach trades cost against reliability, and mature programs combine them rather than relying on one.

LLM-as-a-judge decomposes an answer into atomic claims and asks a strong model whether each claim is entailed by the retrieved context. This scales to thousands of questions, but judges show position bias, verbosity bias and self-preference, so their scores must be calibrated against a human-labeled sample before teams trust them.

MethodStrengthMain weakness
Expert human reviewHighest validity for nuanced answersSlow, costly, limited sample size
Lexical metricsCheap, reproducible, deterministicMiss paraphrases and correct rewording
Embedding similarityTolerates paraphraseRewards fluent but wrong answers
LLM judgeScales, handles open-ended answersBiases, drift when the judge model changes
Scoring methods compared

Agreement between the judge and human raters is commonly reported with Cohen’s kappa or simple percentage agreement. When agreement is weak on a category of questions, that category should stay under human review.

Building a reliable evaluation dataset

A RAG evaluation is only as good as its test set, which should mirror the real distribution of user questions and the real document corpus. Public benchmarks are useful for comparing retrievers, but they rarely reflect a specific organization’s vocabulary, document formats or edge cases.

  • Sample real user queries from logs and remove personal data before labeling.
  • Include unanswerable questions to test whether the system refuses instead of inventing an answer.
  • Add multi-hop questions that require combining two or more passages.
  • Generate synthetic questions from documents to widen coverage, then review a subset by hand.
  • Freeze a versioned golden set so that scores remain comparable across releases.

A few hundred carefully labeled questions usually reveal more than thousands of noisy ones. The set should be refreshed when the corpus changes, because stale gold answers silently penalize a system for citing newer, correct documents.

Diagnosing failures from evaluation results

The practical value of RAG evaluation lies in mapping a metric pattern to a root cause. Reading retrieval and generation scores together narrows the search for the fault to one component of the pipeline.

SymptomLikely causeTypical fix
Low recall, high faithfulnessRetriever misses evidenceHybrid search, query rewriting, better chunking
High recall, low faithfulnessGenerator ignores contextStricter prompt, citations, stronger model
High recall, low precisionToo much irrelevant contextReranking, smaller k, metadata filtering
Good scores, poor user feedbackTest set does not match trafficResample queries from production logs
Scores shift without code changeJudge model or corpus driftPin judge version, re-baseline
Symptom, likely cause and fix

Evaluation should run in continuous integration for every change to embeddings, chunk size, reranker or prompt, and in online monitoring on sampled production traffic. Offline gains that do not appear online usually point to a dataset mismatch.

Limitations and common mistakes

RAG evaluation has inherent limits that experienced practitioners account for. No single score captures quality, and an aggregate number can hide a severe failure on a small but critical class of questions, such as legal or medical queries.

  • Faithfulness is not correctness: an answer can be fully supported by an outdated or wrong document.
  • Overfitting to the golden set happens when teams tune prompts until scores rise without real improvement.
  • Ignoring latency and cost produces systems that score well but are too slow to deploy.
  • Reporting averages only hides variance; per-category breakdowns and confidence intervals are more informative.

A non-technical view

A RAG system works like a research assistant who first pulls books from a library shelf and then writes a short answer based on them. RAG evaluation is the grading of both steps. One grade asks whether the assistant picked the right books. The other asks whether the written answer truly matches what those books say.

Grading the steps separately matters because the cure is different. If the wrong books were chosen, the search must improve. If the right books were chosen but the answer drifted, the writing step must improve. Separate grades point to the right repair.

A concrete example

An internal support team at a software company deploys a RAG assistant over 12,000 help articles. Users complain that some answers sound confident but are wrong. The team builds a golden set of 400 real questions, each labeled with the articles that contain the answer, including 40 questions the documentation cannot answer.

The first run shows recall at 5 of 0.62 and faithfulness of 0.91. The generator is mostly honest, so the problem is retrieval. After adding hybrid search with BM25 and a cross-encoder reranker, recall at 5 rises to 0.84 and end-to-end correctness, checked by an LLM judge calibrated on 100 human-graded answers, improves from 0.58 to 0.76. Refusals on unanswerable questions also rise, and support tickets about wrong answers drop noticeably over the following month.