A stop sequence is a string of characters that, once it appears in the generated output, makes the inference system halt text generation immediately. In language models, it is one of the main controls over where a response ends, working alongside the end-of-sequence token learned during training and the max tokens limit set by the caller.
Stop sequences matter wherever generated text feeds into software rather than a human reader. In applied artificial intelligence, they keep a model from inventing the next turn of a dialogue, from running past the end of a structured record, or from continuing a few-shot pattern indefinitely. They are cheap, deterministic and transparent, but they act on text, not on meaning, so they must be chosen with care.
How a stop sequence ends generation
A stop sequence ends generation by being matched against the decoded output after every new token. An autoregressive model produces one token at a time through next-token prediction; the serving layer decodes each token to text, appends it to a buffer and checks whether the buffer now ends with any configured stop string. When it does, decoding stops and no further tokens are sampled.
The match is usually performed on text rather than on token IDs, because one string can be tokenized in several ways. A stop string such as \nUser: may arrive as one token, as three tokens, or split across a boundary that also contains other characters. Serving engines therefore keep a small lookahead buffer and withhold the last few characters from a streaming client until they are sure those characters are not the beginning of a stop sequence.
Most interfaces exclude the stop sequence from the returned text and report a finish reason such as stop, while some return the matched string in a separate field. This distinction matters for parsers: code that expects a closing delimiter must add it back when the API removes it.
The table below shows the stages of a single generation step and the role the stop sequence plays in each.
| Stage | What happens | Role of the stop sequence |
|---|---|---|
| Forward pass | Model computes logits for the next token | None; the model is unaware of the stop list |
| Sampling | A token is selected by greedy decoding or sampling | None, unless constrained decoding is used |
| Detokenization | The token is converted to text and appended | Prepares the text that will be matched |
| Stop check | The output tail is compared with each stop string | Ends generation on a match |
| Streaming release | Safe characters are sent to the client | Holds back a possible partial match |
Stop sequences compared with other termination controls
A stop sequence is one of three complementary ways to end a response, and each answers a different question. The end-of-sequence token reflects what the model learned about natural endings, the max tokens limit caps cost and latency, and the stop sequence encodes an application rule about where useful output ends.
The following comparison clarifies who controls each mechanism and how it typically fails.
| Mechanism | Who sets it | Typical finish reason | Main failure mode |
|---|---|---|---|
| End-of-sequence token | Model training and chat template | stop or end_turn | Model never emits it and rambles |
| Stop sequence | Application developer | stop or stop_sequence | Fires too early on legitimate text |
| Max tokens | Application developer | length or max_tokens | Truncates an answer mid-sentence |
| Constrained decoding | Grammar or schema | stop when the grammar closes | Grammar too strict for the task |
In chat-tuned models, the chat template already inserts special end-of-turn tokens, so explicit stop sequences are needed less often than with a base model. They remain essential for raw completion endpoints, for few-shot prompting and for any format the model was not trained to close on its own.
Common stop sequence patterns
Effective stop sequences are strings that should never appear inside a correct answer but almost always follow one. Choosing them means looking at the prompt format and asking what the model would write next if it kept going.
| Use case | Example stop string | What it prevents |
|---|---|---|
| Dialogue transcript | \nUser: | The model writing the next user turn |
| Few-shot list | \n\nInput: | Generating extra invented examples |
| Single-line answer | \n | Explanations after a short value |
| Tagged output | </answer> | Text after the closing tag |
| Code completion | \ndef or \nclass | A second, unrequested function |
| Tool-call format | Observation: | The model hallucinating a tool result |
The last pattern is central to agent loops built on the ReAct style: the model writes an action, the stop sequence halts it before it fabricates an observation, and the orchestrator runs the real tool and appends the genuine result. Without that stop, the model would simply invent plausible tool output, a form of hallucination that is hard to detect downstream.
Configuring stop sequences in practice
Stop sequences are passed as a list of strings in the request, commonly under a parameter named stop or stop_sequences. Providers impose a small maximum count, often between four and a few dozen strings, and sometimes a maximum length per string, so the list should contain only the strings that matter.
- Match the exact whitespace of the prompt format;
User:and\nUser:behave very differently. - Prefer distinctive delimiters such as
###or closing tags over common words that can occur naturally. - Always set max tokens as well, because a stop sequence that never appears offers no protection.
- Check the finish reason in every response to distinguish a clean stop from truncation.
- Test with streaming enabled, since partial-match buffering can change perceived latency.
Decoding parameters such as temperature, top-p sampling and repetition penalty do not change how stop sequences are evaluated, but they change how likely the model is to produce the stop string at the right moment. A higher temperature makes unexpected continuations, and therefore missed or premature stops, more frequent.
Limitations and common mistakes
The central limitation is that a stop sequence is a literal string match with no understanding of context. It cannot tell a delimiter from the same characters inside a quotation, a code block or a URL, and it cannot stop generation for a reason that is not visible in the text.
| Symptom | Likely cause | Fix |
|---|---|---|
| Empty or one-word output | Stop string matches the first token produced | Use a longer, more specific delimiter |
| Answer cut in the middle | Stop string occurs naturally in the content | Choose a string that cannot appear in valid output |
| Model keeps going | Whitespace or casing differs from the stop string | Copy the delimiter exactly from the prompt |
| Parser fails on output | API removed the closing delimiter | Re-append the delimiter before parsing |
| Partial delimiter shown to user | Client streams without a lookahead buffer | Buffer the stream tail before display |
Stop sequences are also not a safety mechanism. They cannot reliably block harmful content, because the model can phrase the same idea in endless ways. For structural guarantees, such as valid JSON, constrained decoding or schema-guided generation is more robust; stop sequences are best treated as a lightweight boundary marker.
Stripped to the essentials
A language model writes text one small piece at a time and does not always know when to quit. A stop sequence is a word or symbol you choose in advance, and the moment the model writes it, the system cuts the model off.
It works like a finish line painted on a running track. The runner does not need to understand the race; once they cross the line, the race is over. The key point is that the line must be placed where the useful part really ends, or the answer gets cut short or runs too long.
Where it is applied
A support team builds an assistant that answers product questions by calling an internal search tool. The prompt uses a simple format in which the model writes Thought:, then Action: with a search query, and the orchestrator later appends Observation: with real search results. The developers set Observation: as a stop sequence and limit each step to 256 max tokens.
Before the stop sequence was added, testing showed the model often writing its own fake observation, complete with invented product specifications, and then answering from that fiction. After the change, every step ends cleanly right before the observation, the orchestrator inserts the genuine search result, and the model continues from verified data. The team also logs the finish reason for each step, which reveals that about one step in fifty ends on the token limit instead, flagging prompts that need shorter reasoning.