Decomposed prompting (DecomP) is a modular prompting technique in which a decomposer prompt breaks a complex task into a sequence of simpler sub-tasks and sends each one to a dedicated sub-task handler: another prompt, a specialized model or a symbolic tool. In prompting for large language models, it treats a solution as a small program of prompt calls rather than a single long reasoning trace.
The approach was introduced by Khot and colleagues in 2022 and presented at ICLR 2023. Its central idea is borrowed from software engineering: separate the control logic from the workers, so that each component can be designed, tested, improved or replaced independently.
How decomposed prompting structures a task
Decomposed prompting works by letting the decomposer emit one sub-task call at a time, executing it, and feeding the result back before the next step is chosen. The decomposer prompt contains few-shot demonstrations written as sequences of question and handler pairs, so the model learns which handler name to invoke and what sub-question to pass.
A controller outside the model parses each call, routes it to the named handler and appends the answer to the running trace. The loop ends when the decomposer emits a termination marker such as an end-of-question token. Because handlers can themselves be decomposed, the method supports hierarchical and recursive decomposition, for example reversing a long list by splitting it into halves.
| Component | What it does | Typical implementation |
|---|---|---|
| Decomposer prompt | Plans the next sub-task and its input | Few-shot prompt with handler calls |
| Sub-task handler | Solves one narrow operation | Dedicated few-shot prompt |
| Symbolic handler | Performs exact or external operations | Retrieval, Python function, calculator |
| Controller | Parses calls and passes results | Short orchestration code |
| Trace | Records calls and answers so far | Text appended to the decomposer context |
Types of sub-task handlers
The strength of DecomP lies in mixing handler types within one solution. A handler only needs a defined input and output, so the decomposer does not care whether a language model, a search engine or ordinary code answers the call.
The following table shows the main handler categories and when each fits. In the original experiments, replacing a weak prompt-based handler with a symbolic retrieval function produced the largest gains on open-domain multi-hop questions.
| Handler type | Best for | Main advantage | Main risk |
|---|---|---|---|
| Few-shot prompt | Narrow linguistic sub-tasks | Focused examples for one skill | Errors still possible |
| Recursive decomposer call | Inputs too long for one step | Scales to arbitrary length | Deep call chains add latency |
| Retrieval function | Fact lookup in a corpus | Grounds answers in documents | Depends on index quality |
| Code function | Arithmetic, sorting, string operations | Exact and deterministic results | Requires correct argument parsing |
| Fine-tuned model | High-volume repeated sub-task | Lower cost per call | Training and maintenance effort |
Decomposed prompting compared with related techniques
Decomposed prompting differs from neighboring techniques mainly in who executes each step. In chain-of-thought prompting, one model writes every step inside a single response; in DecomP, steps are delegated to separate handlers with their own prompts or code.
| Technique | How steps are produced | Key difference from DecomP |
|---|---|---|
| Chain-of-thought | One model reasons in a single output | No modular handlers or tools |
| Least-to-most prompting | Decompose first, then solve in order | Fixed plan, one shared solver prompt |
| Self-ask prompting | Model asks and answers follow-up questions | Usually one handler, optionally search |
| Program-aided language model | Model writes code that an interpreter runs | Whole solution expressed as one program |
| Plan-and-solve prompting | Model writes a plan, then executes it | Plan and execution in the same prompt |
DecomP can be seen as a generalization of least-to-most prompting: it adds interleaved planning, specialized handlers and recursion. It also anticipates later agent and tool-calling patterns, where a model chooses functions from a defined catalog.
Designing a decomposed prompting pipeline
A good DecomP design starts by identifying the sub-tasks on which a single prompt fails, then giving each one a handler with a clear contract. Teams that decompose too finely pay for many model calls without a gain in accuracy.
- Run the single-prompt baseline and collect its failure cases.
- Name the recurring operations behind those failures, such as lookup, extraction or arithmetic.
- Define each handler contract with an exact input and output format.
- Write decomposer demonstrations that show realistic call sequences.
- Evaluate each handler in isolation before testing the full pipeline.
The decision table below helps judge whether the technique is worth its overhead for a given workload. The number of model calls typically grows linearly with the number of sub-steps, so latency and cost rise accordingly.
| Situation | Recommendation | Reason |
|---|---|---|
| Multi-hop questions over documents | Use DecomP with retrieval handlers | Each hop needs grounded evidence |
| Exact symbolic operations | Use DecomP with code handlers | Models are unreliable at exact manipulation |
| Short single-step questions | Use a single prompt | Orchestration overhead brings no benefit |
| Strict latency budget | Prefer chain-of-thought | Sequential calls add delay |
Limitations and failure modes
Decomposed prompting introduces new failure points at the boundaries between components. A wrong decomposition sends correct handlers the wrong questions, and errors propagate because later steps trust earlier answers.
- Planning errors: the decomposer skips a needed step or calls an unsuitable handler.
- Format mismatch: a handler returns text the controller cannot parse.
- Error cascades: one wrong intermediate answer corrupts every dependent step.
- Higher cost and latency: many sequential calls replace one response.
- Maintenance burden: each handler prompt needs its own tests and versioning.
Mitigations include validating handler outputs against a schema, adding a verification handler for critical steps, and logging the full trace so failures can be attributed to a specific component instead of the whole system.
Put simply
Decomposed prompting is like a project manager who splits a big job into small tasks and hands each one to the right specialist. One prompt plans the work, other prompts or tools do the individual pieces, and the results are put together at the end.
This helps because an AI model that struggles with a long, tangled request often does well on small, clearly defined questions. The price is that the system needs more steps, so it is slower and costs more to run.
In practice
A support team at a software company needs answers to questions such as which product release fixed a bug that a customer reported against a specific version. A single prompt often guesses, because the facts are spread across release notes, bug reports and version tables.
The team builds a DecomP pipeline with three handlers: a retrieval function over the bug tracker, a retrieval function over release notes, and a small code function that compares version numbers. The decomposer first finds the bug identifier, then looks up its fix, then checks which release contains that fix. On a test set of 200 historical questions, the decomposed version answers clearly more questions correctly than the single prompt, and each wrong answer can be traced to the failing step. The trade-off is an average of four model calls per question instead of one.