Many-shot prompting is a prompting technique in which a language model receives hundreds or even thousands of input-output examples inside a single prompt, so that it infers the task from demonstrations rather than from instructions alone. Also called many-shot in-context learning, it extends classic few-shot prompting to the scale made possible by long context windows of hundreds of thousands to millions of tokens.
Within artificial intelligence, the technique matters because it occupies the space between few-shot prompting and fine-tuning: it can adapt a general model to a specialized task with substantial accuracy gains, yet it requires no change to model weights and no training infrastructure.
How many-shot prompting works
The mechanism is in-context learning: the model conditions its next-token predictions on every token in the prompt, including the demonstrations. With a handful of examples, the model mostly recognizes which known task is being requested. With hundreds of examples, it receives enough evidence to pick up finer decision boundaries, label conventions and output formats that a short prompt cannot express.
Research published in 2024 on long-context models showed that accuracy on tasks such as translation into low-resource languages, summarization, classification and mathematical reasoning often keeps improving as the number of shots grows into the hundreds. The same work found that very large demonstration sets can override biases learned in pretraining, for example when labels are deliberately flipped, which few-shot prompts rarely achieve.
The gains are not uniform. Some tasks plateau after a few dozen examples, and some show a decline at the largest counts, so the effective number of shots must be measured per task rather than assumed. The table below places many-shot prompting on the spectrum of example counts.
| Regime | Number of examples | Main effect on the model |
|---|---|---|
| Zero-shot prompting | None | Relies on instructions and pretraining |
| One-shot prompting | One | Shows format and expected style |
| Few-shot prompting | About 2 to 30 | Identifies the task and output pattern |
| Many-shot prompting | About 50 to several thousand | Learns task-specific nuances and labels |
Variants of many-shot prompting
The main obstacle is the supply of high-quality labeled examples, especially for reasoning tasks that need written solutions. Two variants address this. Reinforced in-context learning replaces human-written rationales with solutions generated by the model and filtered for correct final answers. Unsupervised in-context learning supplies only the problems, without solutions, and relies on the model to activate relevant knowledge.
| Variant | Content of each example | Best suited to |
|---|---|---|
| Standard many-shot | Human-labeled input and output | Classification, extraction, translation |
| Reinforced ICL | Input plus verified model-generated rationale | Math and multi-step reasoning |
| Unsupervised ICL | Input only, without answers | Domains the model partly knows |
| Retrieved many-shot | Examples selected by similarity to the query | Large, diverse example pools |
The retrieved form overlaps with dynamic few-shot prompting, which also selects examples per query. The difference is scale: many-shot designs keep a large fixed or semi-fixed block of demonstrations, which also makes the prompt prefix suitable for caching.
How it compares with few-shot prompting and fine-tuning
Many-shot prompting is a trade between flexibility and cost per request. Fine-tuning pays the cost of adaptation once and then serves short prompts, while many-shot prompting pays for a long prompt on every call. Few-shot prompting is cheap but captures less of the task.
| Property | Few-shot prompting | Many-shot prompting | Fine-tuning |
|---|---|---|---|
| Examples needed | A handful | Hundreds to thousands | Hundreds to many thousands |
| Weight changes | None | None | Model weights updated |
| Cost per request | Low | High without caching | Low after training |
| Time to update | Minutes | Minutes | Hours to days |
| Typical accuracy gain | Modest | Often substantial | Often highest on narrow tasks |
A practical rule follows from this comparison: many-shot prompting suits rapid iteration, closed models without tuning access and tasks that change often, whereas fine-tuning wins when the task is stable and request volume is very high.
Designing an effective many-shot prompt
Good many-shot prompts are curated datasets, not dumps of available records. Examples should cover the full range of labels and edge cases in realistic proportions, use one consistent format and contain no contradictions, because the model copies inconsistencies as faithfully as it copies good patterns.
- Balance the label distribution so the model does not learn a majority-class shortcut.
- Use clear delimiters between examples and a fixed field order.
- Place the instruction and the query after the examples, close to the generation point.
- Shuffle and test example order, because order still affects results.
- Cache the static prefix where the provider supports prompt caching to cut cost and latency.
Evaluation should vary the shot count, for example 25, 100, 250 and 500 examples, on a held-out test set. The resulting curve shows where accuracy saturates and prevents paying for tokens that add nothing.
Limitations and security risks
The most visible limits are cost, latency and context capacity. A prompt with 500 examples can easily reach 100,000 tokens or more, which raises the price of each call and the time to first token. Long contexts also suffer from uneven attention, so examples in the middle of the prompt may influence the model less than those near the ends.
The same mechanism creates a known attack surface. Many-shot jailbreaking fills a long prompt with fabricated dialogues in which an assistant complies with harmful requests, and the probability of a harmful answer rises with the number of such dialogues. Applications that let users insert long text into prompts need input filtering and output monitoring.
| Symptom | Likely cause | Fix |
|---|---|---|
| Accuracy drops at high shot counts | Noisy or redundant examples | Deduplicate and clean the example set |
| One label dominates outputs | Imbalanced demonstrations | Rebalance labels across examples |
| Results change between runs | Order sensitivity | Fix and test a stable ordering |
| Costs exceed budget | Long prompt on every call | Prompt caching or fine-tuning |
A non-technical view
Imagine training a new employee to sort customer letters. You could explain the rules in a few sentences, or you could hand over a binder with five hundred letters that are already sorted and say: do it like this. The binder usually teaches more, because it shows all the tricky cases.
Many-shot prompting does exactly that for an AI model. Instead of a short instruction, the model reads a very large set of solved examples right before the new question. Nothing about the model changes permanently; it simply works from the binder each time.
A concrete example
An insurance company routes incoming claim descriptions to one of 24 internal categories. A zero-shot prompt with category definitions reaches about 71 percent accuracy on a test set of 1,000 claims, mostly because several categories overlap in subtle ways that the definitions do not capture.
The analytics team assembles 400 labeled historical claims, balanced so that every category has at least 12 examples, and places them before each new claim. The prompt grows to roughly 90,000 tokens, and the team uses prompt caching so that only the new claim is processed at full price.
Accuracy rises to about 88 percent, close to a fine-tuned model the team had considered, while category definitions can still be changed by editing examples. Testing at 100 and 200 examples showed most of the gain by 200 shots, so the team later trimmed the prompt to reduce latency.