Self-consistency

Also known as: self-consistency decoding

On this page 7

Self-consistency is a prompting and decoding technique in which a language model generates several independent reasoning paths for the same question and the final answer is chosen by majority vote over their conclusions. Also called self-consistency decoding, it replaces the single greedy answer of standard chain-of-thought prompting with an aggregate that is usually more accurate on reasoning tasks.

The method was introduced by researchers at Google in 2022 and quickly became a reference baseline in prompting research. Its core intuition is simple: a hard problem can be solved by many valid lines of reasoning, but wrong answers tend to scatter while correct answers tend to converge.

How self-consistency works

Self-consistency runs in three steps. First, the model receives a chain-of-thought prompt, either with worked examples or with an instruction to reason step by step. Second, the same prompt is sent several times with stochastic sampling enabled, so each completion follows a different path. Third, the final answers are extracted and the most frequent answer wins.

The technique effectively marginalizes out the reasoning: the individual explanations are discarded, and only the distribution of answers matters. The table below breaks the procedure into its components and typical settings.

ComponentTypical settingPurpose
Reasoning promptFew-shot or zero-shot chain of thoughtElicits explicit intermediate steps
Sampling temperatureAbout 0.5 to 1.0Produces diverse reasoning paths
Number of samples5 to 40 completionsTrades accuracy against cost
Answer extractionParsing a number, label or optionMakes answers comparable
Aggregation ruleUnweighted majority voteSelects the most consistent answer
Components of the self-consistency procedure

In the original experiments, weighting each vote by the model’s own probability gave little advantage over plain majority voting, which is why the unweighted vote remains the common default.

Why sampling many paths improves accuracy

Greedy decoding commits to the single most likely token at each step, so one early mistake propagates to the final answer. Sampling explores alternative paths, and when errors are roughly independent, the chance that most paths share the same wrong answer is low. This is the same logic that makes ensembles more reliable than single models.

Accuracy typically rises quickly with the first 5 to 10 samples and then flattens, while cost keeps growing linearly. Published results on arithmetic benchmarks such as GSM8K showed double-digit percentage-point gains for large models, while improvements on tasks that need little reasoning were small. Gains also depend on model size: weak models produce too few correct paths for the vote to recover.

A useful by-product is a built-in confidence signal. The share of samples that agree with the winning answer correlates with correctness, so a low agreement rate can trigger human review, a stronger model or a refusal to answer.

Variants of self-consistency

Several variants adapt the idea to settings where a simple vote is impractical. The table below lists the main variants and when each one applies.

VariantHow answers are aggregatedWhen it applies
Majority-vote self-consistencyMost frequent extracted answerNumeric, multiple-choice or short answers
Universal self-consistencyThe model selects the most consistent responseFree-form text such as summaries
Weighted votingVotes weighted by likelihood or a verifier scoreWhen a reliable scorer is available
Adaptive samplingSampling stops once agreement is clearCost-sensitive production systems
Self-consistency variants

Adaptive approaches are especially practical: if the first three or four samples agree, further samples rarely change the result, so the system can stop early and reserve heavy sampling for genuinely ambiguous questions.

Self-consistency is a decoding-time aggregation strategy, not a new way of structuring reasoning, so it combines well with other techniques. The comparison below shows how it differs from close neighbors in the prompting toolbox.

TechniqueCore ideaDifference from self-consistency
Chain-of-thought promptingOne reasoning path before the answerSelf-consistency samples many such paths
Tree of thoughtsSearch over partial thoughts with evaluationExplores branches step by step, not whole paths
Self-refineThe model critiques and revises its outputSequential improvement instead of parallel voting
Chain-of-verificationPlanned checks of factual claimsTargets factual errors rather than reasoning variance
Prompt ensemblingDifferent prompts, combined answersVaries the prompt rather than the sampling
Comparison with neighboring techniques

Many modern reasoning models already explore multiple solution paths internally, which can reduce the marginal benefit of external voting. Even so, self-consistency remains a strong and transparent baseline for evaluating whether extra inference compute improves answers.

Limitations and practical cautions

The most obvious drawback is cost and latency: twenty samples mean roughly twenty times the output tokens, although parallel requests can hide most of the delay. Prompt caching reduces input costs but not generation costs.

Teams should therefore measure the accuracy curve on a representative validation set before choosing a sample count. Plotting accuracy against the number of samples usually reveals a clear knee, and settings beyond that point add expense without a meaningful quality gain. The same evaluation should check whether agreement rates are well calibrated for the specific task.

  • Answers must be comparable. Voting fails when outputs are long or open-ended unless they are normalized or judged by a model.
  • Correlated errors survive the vote. If a misconception is built into the prompt or the model, most samples repeat it, and the majority is confidently wrong.
  • Temperature needs tuning. Too low gives near-identical samples; too high produces incoherent reasoning.
  • Parsing errors distort results. Inconsistent answer formats split votes, so a strict output format instruction is essential.

In everyday language

Self-consistency means asking an AI the same question several times and trusting the answer it gives most often. Each time, the AI works through the problem in a slightly different way, so its mistakes tend to differ from one attempt to the next.

It resembles a classroom where several students solve the same math problem on their own. If most of them reach 42, that answer is probably right, even though each student wrote a different solution. The price is extra work, because the question is answered many times instead of once.

Seen in the real world

An insurance company uses a language model to read claim descriptions and assign each claim to one of twelve damage categories. With a single chain-of-thought answer, about one claim in ten lands in the wrong category, and reviewers cannot tell which ones are doubtful.

The engineering team switches to self-consistency with seven samples at temperature 0.7 and a strict instruction to finish with the category code. Claims where at least six samples agree go straight to processing, while claims with weaker agreement go to a human reviewer.

In a pilot of several thousand claims, misclassifications among automatically processed claims fall noticeably, and only about 15 percent of claims need review. The team accepts the roughly sevenfold generation cost because each avoided misrouting saves far more in manual handling time.