Few-shot prompting is the technique of steering a language model by including a small number of worked examples, usually two to ten input-output pairs, in the prompt before the actual query. In the field of language models and artificial intelligence more broadly, it is the most common practical form of in-context learning: the model infers the task, the output format and the expected style from the examples alone, and no model weights are updated.
The technique became widely known when large autoregressive models showed that a handful of demonstrations could replace task-specific fine-tuning for many problems. It sits between zero-shot prompting, which gives instructions without examples, and fine-tuning, which changes the model itself, and it remains a core tool of prompt engineering.
How few-shot prompting works inside the model
Few-shot prompting works because a decoder-only model predicts each next token conditioned on everything in its context window. When the context contains several consistent demonstrations, the most probable continuation after a new input is an output that follows the same pattern. The model is not memorizing the examples; it uses them as evidence about which task is being performed.
Research on in-context learning suggests that demonstrations mainly communicate the label space, the input distribution and the output format, and somewhat less the exact input-output mapping. Experiments with deliberately wrong labels have shown that large models often still perform reasonably, which indicates that format and task recognition carry much of the effect. Larger models, however, follow the actual mapping more closely and can even learn arbitrary or flipped labels from the examples.
A typical few-shot prompt has four parts. The following table shows each component and its function.
| Component | Purpose | Typical content |
|---|---|---|
| Instruction | States the task explicitly | One or two sentences describing the goal |
| Demonstrations | Show the input-output pattern | Two to ten labeled example pairs |
| Delimiters | Separate examples and fields | Labels such as Input and Output, or XML-style tags |
| Query | Supplies the new case to solve | The real input followed by an empty output slot |
Few-shot, zero-shot and one-shot variants
The variants differ only in how many demonstrations appear before the query. The choice affects accuracy, consistency of format, token cost and latency, so it should be made deliberately rather than by habit.
| Variant | Examples | Main strength | Main cost |
|---|---|---|---|
| Zero-shot prompting | 0 | Shortest prompt, no curation | Format and edge cases less predictable |
| One-shot prompting | 1 | Fixes output format cheaply | Model may copy surface details |
| Few-shot prompting | 2 to about 10 | Good balance of accuracy and format | More tokens and careful selection |
| Many-shot prompting | Dozens to hundreds | Approaches fine-tuned quality on some tasks | Requires a long context and high cost |
Modern instruction-tuned models handle many tasks well zero-shot, so few-shot prompting is now most valuable for strict output formats, unusual labeling schemes, domain-specific style and tasks where the instruction alone is ambiguous. With long-context models, many-shot prompting with hundreds of examples has become feasible and can close much of the gap to fine-tuning, at a substantial cost in tokens.
Few-shot prompting also combines with chain-of-thought: each demonstration includes the intermediate reasoning, not only the final answer. This few-shot chain-of-thought form improved arithmetic and logic results on earlier models, although dedicated reasoning models often perform as well or better with plain instructions.
Designing effective demonstrations
The quality of a few-shot prompt depends far more on which examples are chosen than on how many are included. Good demonstrations are representative, varied and formatted exactly as the desired output, because the model imitates every visible regularity, including unintended ones.
- Cover the label space: include at least one example of each class or output type that the task can produce.
- Vary the inputs in length, wording and difficulty so the model does not anchor on one surface form.
- Include a hard case, such as an ambiguous input or an empty result, to show how edge cases are handled.
- Keep formatting identical across all examples, including punctuation, field order and capitalization.
- Retrieve similar examples dynamically from a labeled pool when inputs vary widely, a method often called example selection by semantic similarity.
The following decision table summarizes when few-shot prompting is the right choice and when another approach serves better.
| Situation | Recommended approach | Reason |
|---|---|---|
| Strict JSON or tabular output | Few-shot plus schema | Examples fix field names and ordering |
| Simple, well-known task | Zero-shot instruction | Examples add cost without gain |
| Custom labels or house style | Few-shot prompting | The pattern is hard to describe in words |
| Thousands of labeled cases and high volume | Fine-tuning | Lower per-request cost and stable behavior |
| Task needs current facts | Retrieval augmentation | Examples teach format, not knowledge |
Known biases and failure modes
Few-shot prompting is sensitive to details that seem irrelevant, and experienced practitioners test for these effects systematically. Example order, label balance and formatting can each shift accuracy by a large margin on the same model and task, and the size of the effect varies between models and tasks.
| Problem | Cause | Mitigation |
|---|---|---|
| Majority label bias | One class dominates the examples | Balance classes across demonstrations |
| Recency bias | The last example weighs most | Shuffle order and test several orderings |
| Surface copying | Model reuses names or phrases from examples | Use varied, neutral example content |
| Format drift | Inconsistent example formatting | Standardize delimiters and field order |
| Example leakage | Test cases resemble demonstrations | Keep evaluation data separate from prompt examples |
Two further limits matter in production. Demonstrations consume context window space and raise the cost and latency of every request, which adds up at high volume. Few-shot prompting also cannot add knowledge that the model lacks: it teaches the shape of an answer, not facts beyond the model’s parametric knowledge, so it does not prevent hallucination on unfamiliar topics.
A frequent misconception is that more examples always help. In practice, accuracy often plateaus after a few well-chosen demonstrations, and poorly chosen extra examples can lower performance. Calibration techniques, such as measuring the model’s bias on a content-free input and correcting for it, can reduce some of these effects.
Few-shot prompting with chat models
With chat-based models, demonstrations are usually placed either inside the system prompt or as simulated user and assistant turns that precede the real request. Simulated turns follow the chat template the model was trained on and often produce the most faithful imitation of the format.
When outputs must be parsed by software, few-shot examples work best together with constrained decoding or a JSON schema, because the examples alone reduce but do not eliminate format errors. A low temperature further improves consistency. Teams should store prompts and examples under version control and evaluate every change on a fixed test set, since a single swapped example can change results.
In simple terms
Few-shot prompting means showing an AI model a few solved examples before asking it to solve a new one. It is like handing a new colleague three completed forms and saying, fill in the fourth the same way. Nobody retrains the colleague; they simply see the pattern and follow it.
The examples tell the model what kind of answer is wanted, how long it should be and how it should look. The examples teach the pattern, not new facts, so the model still needs to know the subject. Choosing clear, varied and consistent examples matters more than adding many of them.
A practical example
A customer support team at a mid-sized software company wants incoming tickets sorted into five categories: billing, login problems, bug report, feature request and other. A zero-shot instruction gives inconsistent results, because the model sometimes invents new category names and often confuses login problems with bug reports.
The team builds a prompt with one demonstration per category plus two ambiguous cases, seven examples in total, each written as a ticket text followed by a single category label. They shuffle the order, set temperature to zero and test the prompt on 400 tickets that were labeled by hand and never used as examples.
Agreement with the human labels rises noticeably compared with the zero-shot version, and the model stops inventing category names. The added examples increase each request by roughly 600 tokens, which the team accepts because the tickets now reach the right queue without manual sorting. When volume later grows tenfold, they use the same labeled set to evaluate fine-tuning as a cheaper alternative.