Few-shot prompting

On this page 7

Few-shot prompting is the technique of steering a language model by including a small number of worked examples, usually two to ten input-output pairs, in the prompt before the actual query. In the field of language models and artificial intelligence more broadly, it is the most common practical form of in-context learning: the model infers the task, the output format and the expected style from the examples alone, and no model weights are updated.

The technique became widely known when large autoregressive models showed that a handful of demonstrations could replace task-specific fine-tuning for many problems. It sits between zero-shot prompting, which gives instructions without examples, and fine-tuning, which changes the model itself, and it remains a core tool of prompt engineering.

How few-shot prompting works inside the model

Few-shot prompting works because a decoder-only model predicts each next token conditioned on everything in its context window. When the context contains several consistent demonstrations, the most probable continuation after a new input is an output that follows the same pattern. The model is not memorizing the examples; it uses them as evidence about which task is being performed.

Research on in-context learning suggests that demonstrations mainly communicate the label space, the input distribution and the output format, and somewhat less the exact input-output mapping. Experiments with deliberately wrong labels have shown that large models often still perform reasonably, which indicates that format and task recognition carry much of the effect. Larger models, however, follow the actual mapping more closely and can even learn arbitrary or flipped labels from the examples.

A typical few-shot prompt has four parts. The following table shows each component and its function.

ComponentPurposeTypical content
InstructionStates the task explicitlyOne or two sentences describing the goal
DemonstrationsShow the input-output patternTwo to ten labeled example pairs
DelimitersSeparate examples and fieldsLabels such as Input and Output, or XML-style tags
QuerySupplies the new case to solveThe real input followed by an empty output slot
Components of a few-shot prompt

Few-shot, zero-shot and one-shot variants

The variants differ only in how many demonstrations appear before the query. The choice affects accuracy, consistency of format, token cost and latency, so it should be made deliberately rather than by habit.

VariantExamplesMain strengthMain cost
Zero-shot prompting0Shortest prompt, no curationFormat and edge cases less predictable
One-shot prompting1Fixes output format cheaplyModel may copy surface details
Few-shot prompting2 to about 10Good balance of accuracy and formatMore tokens and careful selection
Many-shot promptingDozens to hundredsApproaches fine-tuned quality on some tasksRequires a long context and high cost
Prompting variants by number of examples

Modern instruction-tuned models handle many tasks well zero-shot, so few-shot prompting is now most valuable for strict output formats, unusual labeling schemes, domain-specific style and tasks where the instruction alone is ambiguous. With long-context models, many-shot prompting with hundreds of examples has become feasible and can close much of the gap to fine-tuning, at a substantial cost in tokens.

Few-shot prompting also combines with chain-of-thought: each demonstration includes the intermediate reasoning, not only the final answer. This few-shot chain-of-thought form improved arithmetic and logic results on earlier models, although dedicated reasoning models often perform as well or better with plain instructions.

Designing effective demonstrations

The quality of a few-shot prompt depends far more on which examples are chosen than on how many are included. Good demonstrations are representative, varied and formatted exactly as the desired output, because the model imitates every visible regularity, including unintended ones.

  • Cover the label space: include at least one example of each class or output type that the task can produce.
  • Vary the inputs in length, wording and difficulty so the model does not anchor on one surface form.
  • Include a hard case, such as an ambiguous input or an empty result, to show how edge cases are handled.
  • Keep formatting identical across all examples, including punctuation, field order and capitalization.
  • Retrieve similar examples dynamically from a labeled pool when inputs vary widely, a method often called example selection by semantic similarity.

The following decision table summarizes when few-shot prompting is the right choice and when another approach serves better.

SituationRecommended approachReason
Strict JSON or tabular outputFew-shot plus schemaExamples fix field names and ordering
Simple, well-known taskZero-shot instructionExamples add cost without gain
Custom labels or house styleFew-shot promptingThe pattern is hard to describe in words
Thousands of labeled cases and high volumeFine-tuningLower per-request cost and stable behavior
Task needs current factsRetrieval augmentationExamples teach format, not knowledge
Choosing between few-shot prompting and alternatives

Known biases and failure modes

Few-shot prompting is sensitive to details that seem irrelevant, and experienced practitioners test for these effects systematically. Example order, label balance and formatting can each shift accuracy by a large margin on the same model and task, and the size of the effect varies between models and tasks.

ProblemCauseMitigation
Majority label biasOne class dominates the examplesBalance classes across demonstrations
Recency biasThe last example weighs mostShuffle order and test several orderings
Surface copyingModel reuses names or phrases from examplesUse varied, neutral example content
Format driftInconsistent example formattingStandardize delimiters and field order
Example leakageTest cases resemble demonstrationsKeep evaluation data separate from prompt examples
Common few-shot failure modes

Two further limits matter in production. Demonstrations consume context window space and raise the cost and latency of every request, which adds up at high volume. Few-shot prompting also cannot add knowledge that the model lacks: it teaches the shape of an answer, not facts beyond the model’s parametric knowledge, so it does not prevent hallucination on unfamiliar topics.

A frequent misconception is that more examples always help. In practice, accuracy often plateaus after a few well-chosen demonstrations, and poorly chosen extra examples can lower performance. Calibration techniques, such as measuring the model’s bias on a content-free input and correcting for it, can reduce some of these effects.

Few-shot prompting with chat models

With chat-based models, demonstrations are usually placed either inside the system prompt or as simulated user and assistant turns that precede the real request. Simulated turns follow the chat template the model was trained on and often produce the most faithful imitation of the format.

When outputs must be parsed by software, few-shot examples work best together with constrained decoding or a JSON schema, because the examples alone reduce but do not eliminate format errors. A low temperature further improves consistency. Teams should store prompts and examples under version control and evaluate every change on a fixed test set, since a single swapped example can change results.

In simple terms

Few-shot prompting means showing an AI model a few solved examples before asking it to solve a new one. It is like handing a new colleague three completed forms and saying, fill in the fourth the same way. Nobody retrains the colleague; they simply see the pattern and follow it.

The examples tell the model what kind of answer is wanted, how long it should be and how it should look. The examples teach the pattern, not new facts, so the model still needs to know the subject. Choosing clear, varied and consistent examples matters more than adding many of them.

A practical example

A customer support team at a mid-sized software company wants incoming tickets sorted into five categories: billing, login problems, bug report, feature request and other. A zero-shot instruction gives inconsistent results, because the model sometimes invents new category names and often confuses login problems with bug reports.

The team builds a prompt with one demonstration per category plus two ambiguous cases, seven examples in total, each written as a ticket text followed by a single category label. They shuffle the order, set temperature to zero and test the prompt on 400 tickets that were labeled by hand and never used as examples.

Agreement with the human labels rises noticeably compared with the zero-shot version, and the model stops inventing category names. The added examples increase each request by roughly 600 tokens, which the team accepts because the tickets now reach the right queue without manual sorting. When volume later grows tenfold, they use the same labeled set to evaluate fine-tuning as a cheaper alternative.