Frequency penalty is a decoding parameter that reduces the logit of each candidate token in proportion to how many times it has already appeared in the generated text, making repeated words progressively less likely. In language models and wider artificial intelligence practice, it is one of the main sampling controls used to curb verbatim repetition and looping without retraining the model.
The parameter acts only at inference time. It does not change the model weights, its knowledge or its reasoning ability; it reshapes the probability distribution over the next token just before sampling, alongside settings such as temperature, top-p sampling and the related presence penalty.
How frequency penalty adjusts token logits
Frequency penalty works by subtracting a count-scaled value from the logits of tokens that already occur in the output. At every generation step, the decoder keeps a tally of how many times each token identifier has been produced. Before the softmax turns logits into probabilities, the penalty subtracts the count multiplied by the penalty coefficient from the logit of each counted token.
In the widely used formulation popularized by commercial completion APIs, the adjusted logit equals the original logit minus count × frequency_penalty minus presence_penalty if the count is above zero. The frequency term is therefore linear in the number of occurrences: a token seen five times receives five times the reduction of a token seen once. Because logits are on a log scale, a reduction of 1.0 divides the token’s unnormalized probability by roughly e, about 2.7.
The table below shows how one fixed coefficient produces a growing penalty as a token repeats.
| Prior occurrences | Logit reduction | Approximate probability factor |
|---|---|---|
| 0 | 0.0 | Unchanged |
| 1 | 0.5 | About 0.61 times |
| 3 | 1.5 | About 0.22 times |
| 6 | 3.0 | About 0.05 times |
The probability factor is approximate because softmax renormalizes over all tokens. A penalty on one token redistributes probability mass to every other candidate, so the actual effect depends on how dominant the penalized token was before the adjustment.
Value ranges and typical settings
Most interfaces accept frequency penalty values between -2.0 and 2.0, with a default of 0, which disables the mechanism. Positive values discourage repetition; negative values encourage repetition, which is rarely useful outside special formats such as lists that must reuse fixed field names.
The following table summarizes common ranges and the behavior each one tends to produce in practice.
| Value range | Typical behavior | Common use |
|---|---|---|
| 0.0 | No adjustment | Code, extraction, factual answers |
| 0.1 to 0.5 | Mild reduction of repeated phrasing | General chat and drafting |
| 0.5 to 1.0 | Noticeably more varied vocabulary | Brainstorming, creative writing |
| Above 1.0 | Avoidance of common words, degraded grammar | Rarely justified |
| Below 0.0 | Increased repetition | Narrow templated formats |
Exact effects differ between models and implementations. Some runtimes count only tokens in the generated output, others also include prompt tokens, and local inference engines often restrict counting to a sliding window of recent tokens, for example the last 64 tokens. These details change how aggressive a given value feels, so settings rarely transfer unchanged between providers.
Frequency penalty compared with related controls
Frequency penalty belongs to a family of anti-repetition and diversity controls that differ in what they measure and how they adjust the distribution. The presence penalty applies a flat, one-time reduction once a token has appeared, which pushes the model toward new topics rather than simply new words. The repetition penalty, introduced with the CTRL model, divides positive logits by a factor such as 1.1 to 1.3, making it multiplicative rather than additive.
| Control | What it measures | Type of adjustment | Main effect |
|---|---|---|---|
| Frequency penalty | Number of prior occurrences | Additive, scales with count | Fewer repeated words and loops |
| Presence penalty | Whether a token has occurred | Additive, flat | Encourages new topics |
| Repetition penalty | Whether a token has occurred | Multiplicative on logits | General anti-repetition |
| Temperature | Nothing, applies globally | Scales all logits | More or less randomness |
| No-repeat n-gram size | Repeated n-grams | Hard block | Prevents exact phrase repeats |
Unlike temperature, frequency penalty is history-dependent: it affects only tokens that have already been generated, so it targets repetition specifically instead of making every choice more random. Unlike a hard n-gram block, it is soft and graded, so a necessary repeated word can still appear if the model rates it highly enough.
Choosing a frequency penalty for a task
The right setting depends on whether repetition is an error or a requirement in the target output. Many tokens must legitimately recur: variable names in code, units in a data table, a product name in a support answer or the keys of a JSON object. Penalizing them can produce inconsistent identifiers and broken structure.
| Situation | Recommended setting | Reason |
|---|---|---|
| Source code generation | 0.0 | Identifiers must repeat exactly |
| Structured JSON output | 0.0 | Keys and syntax recur by design |
| Long-form article drafting | 0.2 to 0.5 | Reduces echoed phrases |
| Brainstorming idea lists | 0.5 to 0.8 | Pushes toward distinct items |
| Small model stuck in loops | 0.3 to 0.6 | Breaks degenerate repetition |
| Translation and summarization | 0.0 to 0.2 | Fidelity matters more than variety |
A sound approach is to change one sampling parameter at a time and evaluate the outputs on a fixed set of prompts. Frequency penalty interacts with temperature and top-p, so raising several of them together makes it hard to see which change caused an improvement or a regression.
Limitations and common mistakes
Frequency penalty is a blunt statistical instrument. It operates on token identifiers, not on meaning, so it cannot detect a paraphrased repetition of the same idea and it may penalize harmless subword pieces that happen to recur, such as common suffixes produced by byte-pair encoding.
- Over-penalizing function words: high values make the model avoid articles and prepositions, which produces stilted or ungrammatical text.
- Treating it as a quality knob: the penalty does not reduce hallucination or improve accuracy; it only changes word choice.
- Ignoring tokenization: the same word with and without a leading space can be different tokens, so counts may not match intuition.
- Copying values across models: modern instruction-tuned models repeat far less than early base models and often need little or no penalty.
- Penalizing required structure: in code and data formats, any positive value can corrupt names, keys or delimiters.
Some recent reasoning-oriented APIs ignore or reject frequency penalty entirely, because their internal decoding is managed by the provider. Checking the parameter support of each model avoids silent no-ops.
Explained without jargon
When a language model writes, it picks one word piece at a time from a list of likely options. Sometimes it gets stuck and keeps choosing the same words, like a speaker who repeats a favorite phrase in every sentence.
Frequency penalty is a small rule that says: the more often you have used a word, the less attractive it becomes. Each repeat makes that word a little harder to pick again. A gentle setting makes the writing feel fresher, while a very strong setting makes the model avoid even ordinary words and the text starts to sound odd. The key point is that it changes word choice, not knowledge.
A typical use case
A product team uses a mid-sized open-weight model to generate first drafts of help center articles of about 600 words each. Reviewers notice that drafts often repeat the same sentence opener, such as a recurring phrase about simple steps, and occasionally fall into a loop that restates the final paragraph twice.
The team runs a small test on 40 fixed prompts. With temperature at 0.7 and frequency penalty at 0, about one draft in eight contains a looped passage. Raising the frequency penalty to 0.4 while keeping every other setting unchanged removes the loops in the test set and reduces repeated openers, while product names and menu labels still appear correctly. A trial at 1.2 produces awkward synonyms for interface terms, so the team settles on 0.4 for articles and keeps 0 for the separate code-snippet pipeline, where identifiers must repeat exactly.