Zero-shot chain-of-thought

Also known as: let's think step by step

On this page 7

Zero-shot chain-of-thought is a prompting technique in which a generic reasoning trigger, most famously the phrase “Let’s think step by step”, is appended to a question so that a language model writes out intermediate reasoning before its answer, without any worked examples in the prompt. In prompting, it is the simplest way to elicit chain-of-thought reasoning, and it is often abbreviated as zero-shot CoT.

The technique was introduced in 2022 in research showing that large language models are zero-shot reasoners: a single instruction, rather than carefully written demonstrations, was enough to raise accuracy sharply on arithmetic, symbolic and commonsense reasoning benchmarks.

How the reasoning trigger works

Zero-shot chain-of-thought works by changing what the model generates first. A model asked a question directly tends to emit an answer immediately, while a model prompted with a reasoning trigger generates a sequence of intermediate steps, and each step becomes context that conditions the next token.

Because a transformer performs a fixed amount of computation per token, writing out steps gives the model more serial computation and a visible scratchpad. Problems that require several dependent operations, such as multi-step arithmetic, benefit most, since each partial result is stored in the text rather than computed implicitly.

The original method uses two prompting stages, summarized below.

StagePrompt additionPurpose
Reasoning extractionQuestion followed by the reasoning triggerGenerate a free-form chain of reasoning
Answer extractionQuestion, reasoning and an answer cueConvert the reasoning into a clean final answer
The two stages of the original zero-shot chain-of-thought method

The answer cue, for example “Therefore, the answer (arabic numerals) is”, forces a parseable final answer. Modern applications often merge both stages into one call by asking for the reasoning followed by a clearly labeled answer line.

Trigger phrases and their effect

The wording of the trigger matters, because models respond to instructive phrases far more than to neutral ones. Early experiments compared many templates and found that phrases encouraging explicit reasoning outperformed misleading or irrelevant text by a wide margin.

Common triggers and their typical role are compared in the following table.

Trigger phraseOriginTypical effect
“Let’s think step by step”Original zero-shot CoT studyStrong baseline for arithmetic and logic
Work it out step by step to be sure the answer is rightAutomatic prompt searchSlightly higher accuracy in some tests
First understand the problem and devise a planPlan-and-solve promptingFewer skipped steps in long problems
“Take a deep breath and work on this problem step by step”Optimizer-generated promptsGains on specific models only
Common reasoning triggers used in zero-shot prompting, paraphrased where needed

Gains from a particular phrasing are model-specific and rarely transfer exactly. Teams should treat any trigger as a hypothesis and confirm it with prompt evaluation on their own data.

Zero-shot chain-of-thought sits at the low-effort end of reasoning techniques. It needs no examples and no extra infrastructure, while related methods trade more prompt work or more model calls for higher accuracy.

TechniqueExamples neededModel callsMain advantage
Zero-shot chain-of-thoughtNoneOne or twoMinimal setup
Few-shot chain-of-thoughtSeveral worked examplesOneControls reasoning format and style
Auto-CoTGenerated automaticallyMany during setupFew-shot quality without manual writing
Self-consistencyOptionalFive to forty samplesMajority vote reduces random errors
Plan-and-solve promptingNoneOneExplicit planning before calculation
Zero-shot chain-of-thought and neighboring techniques

In practice these methods combine well. Self-consistency, for example, is often applied on top of zero-shot CoT: the same triggered prompt is sampled several times and the most frequent final answer is selected.

Few-shot chain-of-thought generally remains more accurate on domain-specific tasks, because demonstrations show the exact reasoning style and answer format expected. Zero-shot CoT is therefore best seen as a strong default baseline: if it already meets the accuracy target, writing and maintaining examples adds cost without clear benefit.

When to use zero-shot chain-of-thought

Zero-shot chain-of-thought is most useful for multi-step problems with a verifiable answer, such as word problems, logic puzzles, date calculations and structured decisions. It adds little to simple factual lookups or short classification, where extra text mainly increases latency and cost.

SituationRecommendationReason
Multi-step arithmetic or logicUse the triggerIntermediate steps reduce skipped operations
Simple fact retrievalSkip the triggerNo reasoning chain to improve
Strict output format neededUse trigger plus answer cueSeparates reasoning from the parsed result
Dedicated reasoning modelUsually skip the triggerReasoning is already built in
High-stakes answersCombine with self-consistency or verificationSingle chains can be confidently wrong
Decision guide for applying a reasoning trigger

Limitations and common misconceptions

Zero-shot chain-of-thought improves average accuracy but does not guarantee correct reasoning. Several limitations are well documented.

  • Scale dependence: the original gains appeared mainly in large models, and small models often produced fluent but wrong chains.
  • Unfaithful explanations: the written reasoning is not necessarily the process that produced the answer, so it should not be read as a reliable audit trail.
  • Error propagation: one early mistake carries through every later step, often with confident wording.
  • Diminishing returns: instruction-tuned and reasoning models already reason by default, so the explicit trigger adds less than it did in 2022.
  • Higher token usage: reasoning text can multiply output length several times, which matters at high request volumes.

A common misconception is that the phrase itself carries special power. The effect comes from shifting the model toward explicit intermediate computation, and any clear instruction with the same intent tends to work similarly.

The idea in brief

Zero-shot chain-of-thought means asking an AI model to show its work before answering, without giving it any solved examples. Adding a short phrase such as “think step by step” makes the model write down each stage of its reasoning.

It resembles a teacher asking a student to solve a math problem on paper rather than in their head. Writing each step makes it harder to skip a calculation, although a careful student can still make a mistake on paper.

A practical scenario

A logistics company uses a language model to answer staff questions about delivery schedules, such as how many working days remain before a shipment arrives when a public holiday falls in between. Direct prompts frequently miscount weekends and holidays.

The team appends a reasoning instruction to every question and asks the model to finish with a single line labeled with the final number. On a test set of 200 historical questions with known answers, accuracy rises from roughly two thirds to above 90 percent, while average response length grows about fourfold.

Because the longer outputs raise costs, the team applies the reasoning trigger only to date and quantity questions detected by a simple classifier, and it keeps short direct prompts for routine lookups such as office addresses.