Program of thoughts

Also known as: PoT

On this page 7

Program of thoughts (PoT) is a prompting technique in which a language model expresses its reasoning as an executable program, usually Python, and an external interpreter runs that program to produce the final answer. The model decides what to compute; the interpreter does the computing.

The method was introduced in 2022 by Wenhu Chen and colleagues for numerical and financial reasoning tasks. In artificial intelligence practice, it addresses a well-known weakness of chain-of-thought prompting: language models often reason correctly in words but make arithmetic errors when they calculate in text.

How program of thoughts separates reasoning from computation

Program of thoughts works by disentangling reasoning from calculation. The prompt asks the model to translate a question into code with meaningful variable names, explicit intermediate steps and a final variable that holds the answer. The code is then executed outside the model, so the result depends on a deterministic interpreter rather than on token prediction.

A typical PoT pipeline has four stages, shown in the table below.

StagePerformed byOutput
Prompt constructionDeveloper or templateInstruction plus code examples
Program generationLanguage modelPython code with named variables
ExecutionSandboxed interpreterNumeric or symbolic result
Answer extractionApplication codeFormatted final answer
Stages of a program of thoughts pipeline

The variable names carry the semantic reasoning. A line such as interest = principal * rate * years documents the logic and performs the calculation at the same time, which makes the program a readable trace of the reasoning. For equations, PoT programs often call a symbolic library such as SymPy to solve them exactly.

This division of labor reflects a structural limitation of transformer models. A model predicts digits as tokens, so long multiplications, divisions and exponents have no reliable internal algorithm, and errors grow with the number of digits. An interpreter has no such limit within its numeric precision, and it handles loops, such as compounding interest over 30 periods, that would require dozens of written steps in chain-of-thought text.

Prompt variants and settings

Program of thoughts can be used in few-shot, zero-shot and ensemble forms, and the choice depends on how much the model already knows about writing task-specific code. Few-shot PoT includes worked question and program pairs; zero-shot PoT relies on an instruction alone.

VariantPrompt contentWhen it applies
Few-shot PoTSeveral question and program examplesDomain-specific formulas or formats
Zero-shot PoTInstruction to write Python onlyStrong code models, generic math
PoT with self-consistencyMultiple sampled programs, majority voteHigh-stakes numeric answers
PoT plus chain-of-thoughtCode for arithmetic, text for interpretationMixed reasoning and lookup tasks
Main program of thoughts variants

The original study reported an average gain of roughly 12 percent over chain-of-thought prompting across math word problem and financial question answering benchmarks, including GSM8K, SVAMP, FinQA and TAT-QA. The size of the gain varies with the model, the dataset and the share of questions that need multi-step arithmetic.

Program of thoughts belongs to a family of methods that delegate exact work to tools, and it is closest to the program-aided language model approach, published at about the same time. Both generate code and execute it; they differ mainly in framing and in how much reasoning stays in natural language comments.

TechniqueReasoning mediumWho computesMain risk
Chain-of-thoughtNatural language stepsLanguage modelArithmetic slips
Program of thoughtsExecutable codeInterpreterWrong program logic
Program-aided language modelCode interleaved with commentsInterpreterWrong program logic
Tree of thoughtsBranching text or statesModel with searchHigh token cost
Self-consistencyMultiple sampled pathsModel, then votingCorrelated errors
How PoT relates to neighboring techniques

Program of thoughts is not a replacement for chain-of-thought in every case. Questions that require commonsense judgment, reading comprehension or ethical reasoning gain little from code, because the hard part is interpretation rather than calculation.

When to use program of thoughts

Program of thoughts is most useful when a task has a precise numeric or symbolic answer that can be derived through a sequence of operations. Financial analysis, unit conversion, date arithmetic, statistics over small tables and physics problems are typical candidates.

The decision table below helps select between PoT and alternative approaches.

SituationRecommended approachReason
Multi-step arithmetic with large numbersProgram of thoughtsInterpreter avoids calculation errors
Iterative or looped computationProgram of thoughtsLoops express repetition exactly
Ambiguous wording, simple mathChain-of-thought firstInterpretation dominates the difficulty
No code execution permittedChain-of-thought with verificationPoT requires a runtime
Answer must be robustPoT with self-consistencyVoting reduces logic errors
Choosing between program of thoughts and alternatives

Limitations, risks and common mistakes

Program of thoughts moves errors from arithmetic to program logic, so a wrong formula still yields a wrong answer, now with false confidence. Running model-generated code also introduces security and operational risks that text-only prompting does not have.

  • Executing code without a sandbox exposes the host to file access, network calls or infinite loops; use isolated runtimes with time and memory limits.
  • Misread problem statements produce correct arithmetic on the wrong quantities, which no interpreter can detect.
  • Floating-point output may need rounding rules, for example currency to two decimal places.
  • Missing answer variables or printed debug output break automatic extraction unless the prompt fixes the output convention.
  • Weak code models generate syntax errors; a retry with the error message often repairs them.

In plain words

Program of thoughts is a way of asking an AI model to write a small calculator program instead of doing the math in its head. The model reads the question and decides which numbers matter. It writes the steps as code. A computer then runs that code and returns the exact result.

It is like a manager who plans the calculation and hands it to an accountant with a reliable spreadsheet. The plan can still be wrong. The arithmetic, however, will be right.

A real-world example

A corporate finance team uses a language model to answer analyst questions about quarterly reports. One question asks for the compound annual growth rate of revenue over five years, given figures of 48.2 million and 71.9 million. With plain chain-of-thought prompting, the model sometimes miscalculates the fractional exponent and returns values that differ in the second decimal place.

The team switches to program of thoughts with a sandboxed Python runtime. The model now writes cagr = pow(end_revenue / start_revenue, 1 / years) - 1, and the interpreter returns 8.33 percent. Reviewers can read each generated program next to the answer, so calculation errors effectively disappear and the remaining mistakes are easy to trace to misread inputs.