Dynamic few-shot prompting, also called dynamic example selection, is a prompting technique in which the demonstrations placed in a prompt are chosen per request from a larger pool, based on how relevant they are to the current input. Instead of a fixed set of examples written once into a prompt template, the system retrieves the examples most similar to the incoming query and inserts them before the task, so a language model sees guidance that fits the case in front of it.
The technique builds on in-context learning, the ability of large language models to infer a task from examples without any change to their weights. It sits between static few-shot prompting and fine-tuning: it adapts behavior to each input, yet the model itself stays untouched.
How dynamic few-shot prompting selects examples
Dynamic few-shot prompting works as a small retrieval pipeline in front of the model. A curated example pool stores input and output pairs, often hundreds or thousands of them. Each example input is converted into a vector embedding and kept in an index. When a new request arrives, it is embedded the same way, and the system picks the top k nearest examples, typically 3 to 8, which are then formatted into the prompt.
The selection step can use semantic similarity, keyword overlap such as BM25, metadata filters (language, product, customer tier) or a combination of these. Many implementations also apply diversity rules, for example maximal marginal relevance, so the chosen examples are not near-duplicates of each other. The ordering matters as well: placing the most similar example closest to the query often improves results, although the effect varies by model.
The table below lists the components of a typical dynamic few-shot pipeline and the role of each.
| Component | Function | Typical choice |
|---|---|---|
| Example pool | Stores verified input and output pairs | Hundreds to thousands of labeled cases |
| Embedding model | Turns text into comparable vectors | General-purpose sentence embedding model |
| Index | Supports fast nearest-neighbor search | Vector database or in-memory index |
| Selector | Chooses which examples to include | Top k by similarity with a diversity rule |
| Prompt template | Formats instructions, examples and query | Fixed instruction block plus example slots |
Selection strategies and key parameters
The quality of dynamic few-shot prompting depends mainly on which examples are selected and how many. Too few examples give weak guidance; too many consume the context window, raise cost and latency, and can push the model toward copying surface patterns. The best number is found empirically on an evaluation set rather than assumed.
The following table compares common selection strategies and their trade-offs.
| Strategy | How it picks examples | Strength | Weakness |
|---|---|---|---|
| Semantic similarity | Nearest embeddings to the query | Strong topical relevance | May return redundant examples |
| Lexical matching | Shared keywords, for example BM25 | Precise for codes and names | Misses paraphrases |
| Maximal marginal relevance | Balances similarity and novelty | Diverse, informative set | Extra tuning parameter |
| Metadata filtering | Restricts pool by category first | Respects hard constraints | Needs well-labeled examples |
| Label-balanced selection | Covers several output classes | Reduces label bias | Less similar examples overall |
Other parameters include the similarity threshold below which no example is used, the token budget reserved for examples, and the format consistency between examples and the expected output. If examples use one output format and the instruction asks for another, models usually follow the examples.
How it compares with related prompting techniques
Dynamic few-shot prompting differs from its neighbors in where the guidance comes from and when it is chosen. One-shot prompting supplies a single fixed example, while many-shot prompting places dozens or hundreds of examples into a long context. Retrieval-augmented generation retrieves facts or documents, whereas dynamic few-shot prompting retrieves demonstrations of the desired behavior.
| Approach | Examples used | Adapts per input | Main cost |
|---|---|---|---|
| Zero-shot prompting | None | No | Lower accuracy on unusual formats |
| Static few-shot prompting | Fixed handful | No | Examples may not fit the query |
| Dynamic few-shot prompting | Retrieved handful | Yes | Retrieval infrastructure and latency |
| Many-shot prompting | Dozens to hundreds | Rarely | Large token usage per call |
| Fine-tuning | Thousands in training | Learned into weights | Training effort and retraining for changes |
Related techniques such as Auto-CoT and active prompting also focus on choosing demonstrations, but they typically build a reusable example set in advance. Dynamic few-shot prompting makes that choice at inference time for every single request.
When to use dynamic example selection
Dynamic few-shot prompting pays off when a task has many distinct input types that each need a different demonstration, and when a library of verified examples already exists or can be collected. Typical cases are text-to-SQL generation, structured data extraction from varied documents, classification with many labels and customer support replies that follow case-specific policies.
| Situation | Recommended choice | Reason |
|---|---|---|
| Uniform inputs, one output format | Static few-shot prompt | Retrieval adds cost without benefit |
| Varied inputs with a large example library | Dynamic few-shot prompting | Relevant guidance for each case |
| Frequent changes to examples or rules | Dynamic few-shot prompting | Update the pool without retraining |
| Stable task with very high volume | Fine-tuning | Shorter prompts lower cost per call |
| Model lacks required facts | Retrieval-augmented generation | Knowledge gaps need documents, not demos |
A useful practice is to log failed outputs, correct them and add them to the pool, which turns production errors into future demonstrations. This feedback loop is one of the main practical advantages of the technique over a fixed prompt.
Limitations and common mistakes
The technique is only as good as its example pool. Incorrect, outdated or inconsistent examples are copied faithfully, and because selection changes from request to request, behavior can vary in ways that are hard to reproduce. Logging the selected example identifiers with each response is therefore essential for debugging and evaluation.
- Similarity is not usefulness: an example that looks similar may demonstrate the wrong reasoning or label.
- Data leakage in evaluation: test cases must be excluded from the pool, or measured accuracy will be inflated.
- Sensitive data exposure: examples drawn from real customer records can reach other users through the prompt.
- Prompt caching suffers: changing examples per request reduces cache hits, so a fixed prefix should come first.
- Retrieval latency adds up: embedding and search add tens of milliseconds or more to each call.
The short version
Language models learn quickly from examples placed in the prompt. Dynamic few-shot prompting simply picks the best examples for each question instead of always showing the same ones.
Imagine a new employee with a thick binder of solved cases. For every new request, a colleague pulls out the three cases most like it and puts them on the desk. The employee works faster and makes fewer mistakes, because the guidance always fits the task at hand. The main point: better-matched examples lead to better answers, without retraining the model.
Where it makes a difference
A retail analytics team lets business users ask questions in English, which a language model turns into SQL queries against a database with about 120 tables. A static prompt with five examples works for sales questions but often fails for inventory and returns, because the examples never show the joins those questions need.
The team builds a pool of about 900 verified question and SQL pairs, embeds every question, and retrieves the five most similar pairs for each new request, with a filter that keeps only examples from the same business area. Every wrong query reported by users is corrected and added to the pool each week.
On a held-out set of 300 questions, the share of queries that run and return correct results rises from roughly 62 percent to roughly 84 percent. Average prompt length stays nearly the same, and the added retrieval step costs about 40 ms per request.