A repetition penalty is a decoding parameter in language models that reduces the logits of tokens that have already appeared in the context, so that the model becomes less likely to select them again. It is applied at inference time, after the network has produced its scores and before sampling or greedy selection, and it changes no model weights.
The technique addresses a well-known failure of autoregressive generation: once a phrase appears, the model often assigns it an even higher probability on the next pass, producing degenerate loops such as the same sentence printed again and again. A modest penalty breaks these loops, while an excessive one damages grammar, terminology and factual accuracy.
How a repetition penalty rescales logits
A repetition penalty works by adjusting the raw score of every previously seen token before the probability distribution is computed. At each step, the decoder collects the set of token IDs present in the prompt and the generated text, or in a recent window of it, and modifies their logits by a factor θ (theta).
In the multiplicative form popularized by the CTRL model in 2019 and adopted by most open-source inference libraries, a positive logit is divided by θ and a negative logit is multiplied by θ. Both operations push the token toward a lower probability, because dividing a positive number shrinks it and multiplying a negative number makes it more negative. A value of θ = 1.0 disables the penalty.
The adjustment is binary with respect to occurrence: a token seen once receives the same penalty as a token seen fifty times. This distinguishes the classic repetition penalty from count-based penalties, which grow with every additional occurrence. The table below summarizes the main parameters that control the mechanism.
| Parameter | Typical values | What it controls |
|---|---|---|
| Penalty factor θ | 1.0 to 1.3 | Strength of the logit reduction |
| Penalty window | 64 to 2,048 tokens | How far back seen tokens are collected |
| Prompt inclusion | On or off | Whether prompt tokens are also penalized |
| Application order | Before temperature and truncation | Which scores the sampler finally sees |
Repetition penalty versus frequency and presence penalties
The repetition penalty is one of several anti-repetition controls, and they differ in arithmetic and in scope. Hosted chat APIs usually expose an additive frequency penalty and presence penalty instead, while local inference engines usually expose the multiplicative repetition penalty.
An additive presence penalty subtracts a fixed amount from every token that has appeared at least once, which makes it the closest relative of the repetition penalty. An additive frequency penalty subtracts an amount proportional to the occurrence count, so frequent tokens are suppressed more strongly than rare ones. The comparison below shows how these methods behave.
| Method | Arithmetic | Depends on count | Typical range |
|---|---|---|---|
| Repetition penalty | Divide or multiply logit by θ | No | 1.0 to 1.3 |
| Presence penalty | Subtract a constant | No | 0 to 2.0 |
| Frequency penalty | Subtract constant times count | Yes | 0 to 2.0 |
| No-repeat n-gram block | Set logit to minus infinity | Exact n-gram match | n = 2 to 4 |
| Sequence-aware penalty | Penalize continuation of a repeated span | Span length | Varies by implementation |
A hard n-gram block guarantees that no sequence of n tokens appears twice, which suits beam search in summarization but can forbid legitimate repeats such as a person’s full name. Sequence-aware samplers target only the extension of an already repeated span, which leaves common single tokens untouched and is therefore gentler on grammar.
Effects of different penalty strengths
The effect of a repetition penalty is strongly non-linear in practice, because function words, punctuation and code syntax appear in almost every context. A small increase in θ can therefore change the output style far more than the number suggests.
| Penalty factor | Observed behavior | Main risk |
|---|---|---|
| 1.0 | Unmodified model distribution | Loops in long or greedy outputs |
| 1.05 to 1.15 | Fewer loops, natural phrasing | Minimal for most tasks |
| 1.2 to 1.3 | Noticeably varied wording | Synonym drift and altered terms |
| Above 1.5 | Avoidance of common words | Broken grammar and incoherence |
The danger at high values comes from the fact that the penalty cannot distinguish harmful repetition from necessary repetition. Articles, variable names, units and the subject of a document must recur, and penalizing them forces the model into unnatural synonyms or invented spellings. In code, an aggressive setting may rename a variable halfway through a function, which produces a program that does not compile.
Configuring a repetition penalty in practice
The right configuration depends on the decoding strategy, the task and the model’s alignment. Instruction-tuned and reasoning models trained with modern preference optimization loop far less than older base models, so they often need no penalty at all or only a very light one.
| Situation | Recommended setting | Reason |
|---|---|---|
| Greedy decoding of long text | 1.1 to 1.2 | Greedy search loops most easily |
| Creative writing with sampling | 1.05 to 1.15 | Adds variety without harming style |
| Code generation | 1.0 or a narrow window | Identifiers must repeat exactly |
| Structured output such as JSON | 1.0 | Keys and brackets must recur |
| Small or older base model | 1.1 to 1.2 with a short window | Weak models repeat more often |
A limited penalty window is one of the most effective refinements. Restricting the penalty to the last 64 to 256 tokens prevents the model from avoiding words simply because they appeared in a long system prompt. Practitioners also tune one anti-repetition control at a time, because stacking a repetition penalty with frequency and presence penalties multiplies their side effects.
Limitations and common mistakes
A repetition penalty treats a symptom rather than the cause of degenerate text. Loops usually indicate a weak model, a poorly formatted prompt, a missing chat template or overly deterministic decoding, and fixing those causes often removes the need for any penalty.
- Penalizing the prompt by default: some implementations include all input tokens, so key terms from the instructions become harder to use in the answer.
- Treating the factor as linear: a step from 1.1 to 1.3 is not a small change, because it affects nearly every common token.
- Ignoring tokenization: the penalty acts on subword tokens, so suppressing a fragment can also suppress unrelated words that share it.
- Assuming it improves accuracy: discouraging repeated facts can push the model toward less likely and sometimes incorrect alternatives.
Stripped to the essentials
When a language model writes text, it chooses one word piece at a time. Sometimes it gets stuck and repeats the same phrase over and over, like a record with a scratch.
A repetition penalty is a small rule that makes words already used a little less attractive for the next choice. A gentle penalty keeps the text fresh. A strong one forces the model to avoid even ordinary words, and the writing becomes strange. The key is a light touch: just enough to stop the loops, not so much that normal language suffers.
Where it is applied
A support team runs a small open-weight model on its own server to draft long product descriptions. With greedy decoding and no penalty, roughly one draft in ten ends with the same feature sentence repeated until the token limit is reached.
The engineers set the repetition penalty to 1.1 with a window of the last 128 tokens, and they exclude the long system prompt from the penalized set. Looping drafts almost disappear, and product names and units still appear consistently. A trial at 1.4 is rejected because the model began replacing the product name with awkward synonyms. The team keeps the lighter setting as the production default.