Chain-of-verification

Also known as: CoVe

On this page 7

Chain-of-verification (CoVe) is a prompting technique that reduces hallucinations by making a language model check its own draft answer before it commits to a final response. In prompting, it belongs to the family of self-correction methods: the model writes a baseline answer, plans verification questions about the facts it stated, answers those questions independently and then produces a revised, verified answer.

The method was described by researchers at Meta AI in 2023. Its central observation is that models often answer a short, focused question correctly even when they made the same fact wrong inside a longer generation. CoVe turns that asymmetry into a procedure, and it requires no fine-tuning, no retrieval and no external tools.

The four steps of chain-of-verification

Chain-of-verification runs as a fixed sequence of four prompts or prompt stages, each with a distinct job. The sequence separates generating claims from checking them, which is the source of its effect.

StepModel taskOutput
1. Baseline responseAnswer the original query normallyA draft that may contain errors
2. Plan verificationsWrite questions that test each factual claimA list of verification questions
3. Execute verificationsAnswer each question on its ownShort, independent answers
4. Final verified responseCompare draft with answers and reviseA corrected final answer
The CoVe procedure step by step

For example, if the draft lists politicians born in a particular city, the planning step produces one question per name, such as where that person was born. Answers that contradict the draft show which items to remove or correct in the final step.

Why independent verification reduces hallucinations

CoVe works because errors in long generations are partly self-reinforcing. When a model answers verification questions while the flawed draft is still in its context, it tends to copy its earlier mistakes, a pattern the original study described as repeating hallucinations. Answering each question without the draft in view breaks that dependency.

The researchers therefore compared several execution variants that differ in how much context the verification step can see. The table shows the variants and their trade-offs.

VariantHow verification runsMain trade-off
JointQuestions and answers in one prompt with the draftCheapest, but repeats draft errors
Two-stepQuestions planned first, answered in a separate promptLess copying, still batched
FactoredEach question answered in its own promptStrongest independence, more calls
Factor plus reviseFactored answers plus an explicit consistency checkBest accuracy, highest cost
Execution variants of chain-of-verification

In the reported experiments, the factored variants performed best on list questions, closed-book question answering and long-form biography generation. Precision on list-based tasks improved substantially over the baseline, and long-form factuality also rose, although exact gains depend on the model, the task and the prompts.

CoVe differs from neighboring techniques mainly in what it checks and how. It targets factual claims through question decomposition, whereas other methods target reasoning paths, writing quality or consensus across samples.

TechniqueCore ideaMain target
Chain-of-verificationVerify claims with independent questionsFactual hallucinations
Self-refineCritique and rewrite the whole outputGeneral quality and style
Self-consistencySample many answers and take the majorityReasoning accuracy
Self-ask promptingPose follow-up questions before answeringMulti-hop questions
Step-back promptingAsk a more general question firstPrinciple-based reasoning
CoVe and neighboring techniques

Self-ask decomposes a question before answering it, while CoVe decomposes the answer after it is written. Self-refine critiques the output as a whole, which makes it vulnerable to the same blind spots that produced the errors; CoVe instead narrows each check to one fact. The methods combine well, for example CoVe for facts and self-consistency for calculations.

Implementing chain-of-verification in practice

A practical CoVe implementation is usually a small orchestration script rather than a single prompt, because the factored variant needs separate model calls. The decision table below helps select a configuration.

SituationRecommended setupReason
Lists of entities or factsFactored, one question per itemEach item can be checked in isolation
Long-form summariesFactor plus reviseCatches contradictions across sections
Tight latency budgetTwo-step with batched questionsFewer calls, partial benefit
Answers needing current dataCoVe with retrieval in step 3Model memory alone may be outdated
Choosing a CoVe configuration

Good verification questions are closed, specific and answerable in a sentence, such as a date, a place or a yes-or-no fact. The final prompt should instruct the model to drop claims that failed verification rather than rephrase them, and to keep the original structure where checks succeeded. Running the verification step at a low sampling temperature keeps answers stable.

Limitations and failure modes

Chain-of-verification reduces but does not eliminate hallucinations, because the model still checks itself with its own knowledge. If the model is confidently wrong about a fact, the verification answer can be wrong in the same way.

  • Higher cost and latency: the factored variant multiplies model calls, often by the number of claims.
  • Coverage gaps: claims the planning step does not turn into questions are never checked.
  • Factual scope only: CoVe does little for flawed reasoning, weak arguments or poor style.
  • Over-correction: a noisy verification answer can remove a correct item from the final response.
  • Knowledge cutoff: without retrieval, verification cannot confirm events the model never learned.

For high-stakes outputs, practitioners therefore pair CoVe with retrieval-based fact checking or human review, and they evaluate it on a fixed test set before assuming that the extra calls pay off.

Understanding it simply

When people write from memory, they sometimes mix up names or dates. A careful writer rereads the draft, picks out each fact and asks a simple question about it, such as when a person was really born. Answering one small question at a time is easier than getting everything right in one long piece.

Chain-of-verification asks an AI model to do the same thing. It writes a draft, quizzes itself on each fact separately and then fixes the draft. The key idea is that small, separate checks catch mistakes that a long answer hides.

Applied in practice

A content team at an educational publisher uses a language model to draft short biographies of historical scientists for a classroom reference series. Early drafts read well but occasionally contain wrong birth years, invented awards or misattributed discoveries, which editors must find by hand.

The team adds a chain-of-verification step. After each draft of about 300 words, the model lists every factual claim as a question, typically 12 to 20 questions per biography, and answers each in a separate call without seeing the draft. A final prompt removes or corrects any claim whose answer disagrees, and flags uncertain items for an editor. On a sample of 50 biographies, the editors find noticeably fewer factual corrections per draft, while generation time per biography rises from seconds to roughly a minute, a trade the team accepts because editing time falls more than generation time grows.