A program-aided language model (PAL) is a prompting technique in which a language model translates a problem into executable code, usually Python, and an external interpreter runs that code to produce the final answer. In prompting for artificial intelligence systems, PAL is the standard example of dividing labor between a model and a deterministic tool: the model handles understanding and decomposition, while the interpreter handles calculation.
The approach was introduced in 2022 research on arithmetic and symbolic reasoning and is also called program-aided prompting. Its central insight is that large models often reason correctly but calculate poorly, so moving the calculation into code removes a major source of error.
How a program-aided language model solves a problem
A PAL pipeline works by asking the model for a program instead of a prose answer. The prompt contains a few worked examples in which each word problem is followed by Python code whose variable names and comments mirror the reasoning steps. The model imitates this pattern for the new question, and the answer is whatever the code returns.
The model therefore never states the final number itself. It declares quantities, relations and operations, and the interpreter performs the arithmetic exactly. The table below traces the stages of a typical run.
| Stage | Performed by | Output |
|---|---|---|
| Few-shot prompt | Developer | Examples pairing questions with code |
| Code generation | Language model | Python function with commented steps |
| Execution | Python interpreter | Computed value or an exception |
| Answer extraction | Application code | Returned value formatted for the user |
| Optional retry | Application code and model | Corrected program after an error |
Code is a useful intermediate language because it is unambiguous and checkable. A variable such as apples_left = apples_bought - apples_eaten exposes the model’s interpretation of the problem, which makes errors visible and debuggable in a way that free-form reasoning text is not.
Where PAL performs best
PAL is strongest on tasks where the difficulty lies in exact computation or state tracking rather than in world knowledge. Benchmarks used to evaluate it include grade-school math word problems, a variant with large, awkward numbers, date arithmetic, object counting and symbolic manipulation.
Gains are largest when numbers are big or when many intermediate values must be tracked, because chain-of-thought text accumulates small arithmetic slips at each step. On problems that need commonsense judgment or open-ended explanation, PAL offers little or no advantage.
| Task type | Benefit from PAL | Reason |
|---|---|---|
| Multi-step arithmetic | High | Interpreter removes calculation errors |
| Date and time arithmetic | High | Libraries handle calendars correctly |
| Counting and state tracking | High | Variables hold exact running totals |
| Symbolic and logic puzzles | Moderate | Depends on clean formalization |
| Commonsense questions | Low | No computation to offload |
| Open-ended writing | None | Output is not a computed value |
PAL compared with related prompting techniques
PAL differs from chain-of-thought prompting because reasoning is written as code and computation is delegated. In chain-of-thought, the model writes and computes everything in natural language. Program of thoughts is a closely related technique published at nearly the same time, with a similar idea and a stronger emphasis on financial and numerical datasets.
| Technique | Reasoning format | Who computes the answer |
|---|---|---|
| Chain-of-thought | Natural-language steps | The language model |
| Program-aided language model | Code with comments | An external interpreter |
| Program of thoughts | Code, often with less prose | An external interpreter |
| Self-consistency | Several sampled reasoning paths | Majority vote over model outputs |
| Tool-calling agents | Structured function calls | Predefined tools or APIs |
These techniques combine well. Sampling several programs and taking a majority vote over executed results applies self-consistency to PAL and usually improves accuracy further. Modern assistants with a built-in code interpreter apply the PAL principle automatically whenever they decide to run code.
Designing an effective PAL prompt
A good PAL prompt makes the model write readable, self-contained programs. Examples should use descriptive variable names, one comment per reasoning step and a single function that returns the answer, because the model copies the structure of the demonstrations closely.
- Provide three to eight worked examples that cover the variety of expected problems.
- Ask for a function with a fixed name, such as
solution(), to simplify answer extraction. - Restrict the program to the standard library or an explicit allowlist of modules.
- Return the raw error message to the model for one or two repair attempts, not unlimited retries.
Instruction-tuned models can often produce PAL-style code with zero examples if the instruction is explicit. Few-shot examples remain valuable, however, when a domain has specific conventions, such as rounding rules, currency handling or units of measurement.
Limitations, risks and failure modes
PAL guarantees correct calculation, not correct understanding. If the model misreads the problem, the interpreter will compute the wrong quantity precisely. Executing model-generated code also introduces a serious security concern, so production systems must run it in an isolated environment.
| Symptom | Likely cause | Remedy |
|---|---|---|
| Exact but wrong answer | Misinterpreted problem | Better examples or self-consistency voting |
| Runtime exception | Invalid or incomplete code | Feed the error back for a repair attempt |
| Program never ends | Unbounded loop | Enforce timeouts and resource limits |
| File or network access | Unsafe generated code | Sandbox with no network or file system |
| Answer in wrong units | Missing domain conventions | State units and rounding in examples |
PAL also adds latency and infrastructure: an interpreter, a sandbox and error handling. For simple questions, a direct answer from a strong model may be cheaper and fast enough, so the technique is best reserved for computation-heavy workloads.
In everyday language
Language models understand questions well. They can still make careless mistakes with numbers. A program-aided language model acts like a sensible student. It writes down the calculation and lets a calculator do the math.
The model does not answer directly. It writes a short piece of code that lists the steps, and a computer runs that code. The main takeaway is simple: the model plans and the computer calculates. The arithmetic is exact, but the plan itself still has to be right.
Seen in the real world
A logistics company builds an internal assistant that answers dispatcher questions such as how many pallets fit on a truck after mixed loads, weight limits and partial returns. Early tests with plain chain-of-thought prompting show that roughly one answer in eight contains an arithmetic slip, usually in multi-step unit conversions.
The team switches to a PAL design. The model receives six examples that convert each question into a Python function with named variables for weights, dimensions and limits, and the generated code runs in a sandbox with a two-second timeout and no network access. Runtime errors are returned to the model for a single repair attempt.
In follow-up testing, arithmetic errors nearly disappear, and the remaining mistakes come from misread questions, which reviewers can spot quickly by reading the variable names in the generated code.