Prompt chaining is a technique for working with language models in which a complex task is decomposed into a sequence of separate model calls, and the output of each call becomes part of the input to the next. Instead of asking one large prompt to research, reason, format and verify at once, the application runs small, focused prompts in a fixed order, often with code between them that parses, validates or transforms intermediate results.
In artificial intelligence engineering, prompt chaining sits between a single prompt and a fully autonomous agent. The control flow is defined by the developer, not chosen by the model, which makes chains predictable, testable and easy to debug. The technique is a core pattern of prompt engineering for production systems built on an LLM.
How a prompt chain passes work between steps
A prompt chain works by turning one goal into a pipeline of narrow subtasks, each with its own instructions, inputs and expected output format. Step one might extract facts from a document, step two might classify them, and step three might draft a summary from the classified facts only. Every step receives only the context it needs, which keeps prompts short and reduces distraction inside the context window.
Between steps, ordinary code acts as glue logic. It parses JSON, checks that required fields exist, filters items, calls external tools or databases, and decides which prompt runs next. This gate between calls is what separates a chain from a long conversation: errors can be caught before they propagate, and a failing step can be retried without repeating the whole task.
The table below lists the components that appear in most production chains and the role each one plays.
| Component | Role in the chain | Typical implementation |
|---|---|---|
| Step prompt | Performs one narrow subtask | Template with variables and a fixed output format |
| Intermediate output | Carries results to the next step | JSON object, list or short text |
| Validation gate | Rejects malformed or implausible output | Schema check, regular expression, rule or second model |
| Router | Selects the next step from the result | Conditional code or a classification prompt |
| State store | Keeps data shared across steps | Variables, cache or database record |
Common prompt chaining patterns
Prompt chaining appears in several recurring shapes, and most real systems combine two or three of them. The simplest is a linear chain, where steps always run in the same order. More advanced designs add branching, parallel work or loops that revise a draft until it passes a check.
| Pattern | Structure | Best suited for |
|---|---|---|
| Linear chain | Fixed sequence of steps | Extraction, transformation and formatting pipelines |
| Conditional routing | A classifier picks one branch | Support tickets, intent handling, mixed input types |
| Map and reduce | Same prompt on many chunks, then a merge step | Long documents beyond one context window |
| Generate and critique | Draft, review, then revise | Writing quality, code review, policy compliance |
| Retrieval chain | Query rewriting, search, then answer | Grounded answers over private documents |
The generate and critique loop deserves care. A reviewing prompt can catch omissions and format errors well, but a model reviewing its own output tends to share the same blind spots, so factual checks work better when the critic receives external evidence or a deterministic test.
Prompt chaining compared with related techniques
Prompt chaining is often confused with chain-of-thought and with agents, although the three differ in who controls the steps and where the reasoning happens. Chain-of-thought keeps all reasoning inside one model response; prompt chaining spreads work across several calls with code in between; an agent lets the model decide its own next action in a loop.
| Approach | Who controls the flow | Number of calls | Main strength |
|---|---|---|---|
| Single prompt | No flow, one response | One | Lowest latency and cost |
| Chain-of-thought | The model, within one answer | One | Better multistep reasoning in place |
| Prompt chaining | The developer, in code | Several, fixed | Predictability and inspectable steps |
| AI agent | The model, in a loop | Variable | Flexibility on open-ended tasks |
A practical rule follows from this comparison: use a chain when the steps are known in advance, and consider an agent only when the path genuinely depends on what the model discovers along the way. Many teams find that a well-designed chain handles most business workflows with fewer failures than an agent.
Designing a reliable prompt chain
A reliable chain starts with clear step boundaries: each prompt should have one job that can be described in a single sentence and evaluated on its own. When a step needs two unrelated skills, splitting it usually raises accuracy more than rewording the instructions.
- Define a strict output format for every step, ideally a JSON schema, and use constrained decoding or structured output where the platform supports it.
- Set temperature low for extraction and classification steps, and allow higher values only in creative drafting steps.
- Pass only the fields the next step needs, rather than the full transcript of earlier calls.
- Log the input and output of every step, so that a wrong final answer can be traced to the step that caused it.
- Choose a model per step: a small language model often handles routing or extraction, while a larger reasoning model handles the hardest step.
Evaluation is easier with chains than with single prompts because each step can have its own test set and metric. A team can measure extraction precision, routing accuracy and summary quality separately, then improve the weakest link instead of tuning one opaque prompt.
Limitations and failure modes of prompt chaining
The main weakness of prompt chaining is error propagation: a mistake in an early step is passed forward as if it were fact, and later steps rarely question it. Per-step accuracy also multiplies, so five steps that each succeed 95 percent of the time give an end-to-end success rate of only about 77 percent without validation.
| Symptom | Likely cause | Fix |
|---|---|---|
| Final answer contains invented facts | Hallucination accepted in an early step | Validate against source text before passing on |
| Parsing errors between steps | Free-form output instead of a schema | Enforce structured output and retry on failure |
| High latency for users | Too many sequential calls | Merge trivial steps, run independent steps in parallel |
| Rising cost per request | Full history copied into every step | Pass compact summaries or selected fields |
| Inconsistent tone or terms | Steps written without shared conventions | Reuse one system prompt and glossary |
Chains also add latency and cost, because every step is a separate request with its own input tokens. For a simple task, a single well-written prompt is often faster and just as accurate, so chaining should be justified by measurable gains in quality or control, not used by default.
The idea in brief
Prompt chaining means breaking a big request into small steps and giving each step to the AI model separately. The answer from one step is handed to the next, much like an assembly line where each station does one job and passes the part along.
Because every station is simple, it is easier to spot where something went wrong and fix only that station. The trade-off is that the whole line takes a little longer and costs a little more than one quick request. The key benefit is control: people decide the order of the work, and they can check each piece before it moves on.
A practical scenario
A mid-sized insurance company receives about 2,000 claim emails a day and wants a short, structured summary of each one for its adjusters. A single prompt that reads the email and writes the summary produces uneven results: policy numbers are sometimes missed and damage types are mixed up.
The engineering team builds a four-step chain. The first prompt extracts fields into JSON, such as policy number, date and claimed amount, and code checks the policy number against the internal database. The second prompt classifies the claim type with a small, inexpensive model. The third prompt drafts the summary using only the validated fields and the original email, and the fourth prompt checks the draft against a list of required elements.
After two weeks of testing on 500 labeled emails, the team reports that missing policy numbers dropped to almost zero because invalid values were caught at the gate, and adjusters spend noticeably less time correcting summaries. Total latency rises to about six seconds per email, which is acceptable for a background process.