A prompt is the input given to a language model: the instructions, context, examples and questions that the model reads before generating its output. For language models, the prompt is the only channel through which a user steers behavior at inference time, without changing the model’s weights. The model converts the prompt into tokens and produces a continuation that is statistically consistent with it.
In modern chat-based systems, a prompt is rarely a single sentence. It is usually a structured sequence of messages, including a system message, prior conversation turns, retrieved documents and tool results, all of which must fit inside the model’s context window. Designing this input deliberately is known as prompt engineering or, more broadly, context engineering.
How a language model processes a prompt
A language model does not read a prompt as a human does. Tokenization first splits the text into tokens, typically fragments of words, and the model then predicts one token at a time, each conditioned on the prompt and on everything generated so far. The output is therefore a continuation of the prompt, shaped by patterns learned during training.
Chat models wrap messages in a chat template with special tokens that mark roles such as system, user and assistant. Instruction tuning and preference training teach the model to treat these roles differently, for example to give system instructions higher priority than user text.
| Part | Content | Purpose |
|---|---|---|
| System message | Role, rules, tone and constraints | Sets persistent behavior for the conversation |
| User message | The request or question | States the immediate task |
| Context | Documents, data, retrieved passages | Grounds the answer in specific facts |
| Examples | Sample inputs with desired outputs | Demonstrates format and style |
| Conversation history | Earlier user and assistant turns | Maintains continuity |
| Output specification | Required format, length or schema | Makes results predictable and parseable |
Prompting techniques and when to use them
Prompting techniques differ in how much guidance they give the model. The right technique depends on task complexity, the need for consistency and the available context budget.
| Technique | How it works | Best suited for |
|---|---|---|
| Zero-shot prompting | Instruction only, no examples | Simple, well-known tasks |
| Few-shot prompting | A handful of input-output examples | Specific formats and classification labels |
| Step-by-step reasoning | Asks the model to work through intermediate steps | Math, logic and multi-step analysis |
| Role or persona prompting | Assigns a perspective or expertise | Tone and domain framing |
| Retrieval-augmented prompting | Inserts retrieved documents as context | Answers grounded in current or private data |
| Prompt chaining | Splits a task into several sequential prompts | Long workflows needing checks between steps |
Few-shot examples are powerful but strong signals: models tend to copy their length, style and even errors. Examples should therefore be diverse and correct, and they should cover edge cases rather than repeat the easiest case.
Prompt design and generation parameters
A prompt determines what the model is asked to do, while sampling parameters determine how the next tokens are chosen. Both affect output quality, and they should be tuned together.
| Concept | What it controls | Relation to the prompt |
|---|---|---|
| Token | Unit of text processing and billing | Prompt length is measured in tokens |
| Context window | Maximum tokens per request | Prompt and output must fit together |
| Temperature | Randomness of token selection | Low values make prompted output more consistent |
| Fine-tuning | Model weights | Changes behavior permanently instead of per prompt |
| Hallucination | Fluent but false output | Grounded, specific prompts reduce but do not remove it |
Effective prompts share several traits: clear and specific instructions, the relevant context placed where the model can use it, an explicit output format, and a statement of what to do when information is missing, such as answering that the data does not say. Delimiting context clearly, for example with headed blocks or tags, helps the model separate instructions from data.
Prompt injection and other risks
Because a language model treats all prompt text as potential instructions, untrusted content inside a prompt can change behavior. This class of attack, called prompt injection, is the main security risk of prompts that include web pages, emails or user files.
| Risk | Cause | Mitigation |
|---|---|---|
| Direct prompt injection | User text tries to override system rules | Instruction hierarchy, input filtering, output checks |
| Indirect prompt injection | Hidden instructions in retrieved content | Treat retrieved text as data; limit tool permissions |
| System prompt leakage | Model reveals its instructions | Keep secrets out of prompts entirely |
| Context overflow | Prompt exceeds the context window | Truncate, summarize or retrieve selectively |
| Prompt brittleness | Small wording changes alter results | Evaluate prompts on test sets before deployment |
No prompt wording fully prevents injection, so applications that let a model take actions should apply the principle of least privilege and require confirmation for consequential steps.
Common mistakes when writing prompts
Many poor results trace back to the prompt rather than the model. The most frequent problems are easy to recognize and correct.
- Vague goals such as asking for something better without saying better for whom or by which criteria.
- Missing context that the author knows but the model cannot, such as the audience, the product or the date.
- Conflicting instructions, for example demanding brevity while requiring exhaustive detail.
- Overloading one prompt with many unrelated tasks instead of chaining smaller prompts.
- Treating one good output as proof; prompts need testing across many varied inputs because outputs are probabilistic.
Prompts in production systems deserve the same discipline as code: version control, test cases and regression checks whenever the prompt or the underlying model changes.
A non-technical view
A prompt is what a person types or sends to a language model to get a response. It can be a short question or a detailed brief with background material and examples.
It works much like briefing a capable new colleague. A vague request gets a generic answer. A clear request with context and a goal gets a useful answer. The model cannot read minds or know unstated facts, so what the prompt says largely decides what comes back.
A concrete example
A legal operations team uses a language model to extract key dates from supplier contracts. The first prompt simply asks the model to list the important dates. Results vary: some answers include page numbers, some miss renewal deadlines, and the format changes from contract to contract.
The team rewrites the prompt. A system message defines the task and states that missing dates must be reported as not found. The prompt adds two worked examples and requires output as JSON with fixed fields for start date, end date, renewal notice period and termination date, and the contract text is placed in a clearly delimited block. Temperature is set low. Tested on 50 contracts, the output becomes consistent and machine-readable, and reviewers now check flagged gaps instead of reformatting every answer.