Frequency penalty

On this page 7

Frequency penalty is a decoding parameter that reduces the logit of each candidate token in proportion to how many times it has already appeared in the generated text, making repeated words progressively less likely. In language models and wider artificial intelligence practice, it is one of the main sampling controls used to curb verbatim repetition and looping without retraining the model.

The parameter acts only at inference time. It does not change the model weights, its knowledge or its reasoning ability; it reshapes the probability distribution over the next token just before sampling, alongside settings such as temperature, top-p sampling and the related presence penalty.

How frequency penalty adjusts token logits

Frequency penalty works by subtracting a count-scaled value from the logits of tokens that already occur in the output. At every generation step, the decoder keeps a tally of how many times each token identifier has been produced. Before the softmax turns logits into probabilities, the penalty subtracts the count multiplied by the penalty coefficient from the logit of each counted token.

In the widely used formulation popularized by commercial completion APIs, the adjusted logit equals the original logit minus count × frequency_penalty minus presence_penalty if the count is above zero. The frequency term is therefore linear in the number of occurrences: a token seen five times receives five times the reduction of a token seen once. Because logits are on a log scale, a reduction of 1.0 divides the token’s unnormalized probability by roughly e, about 2.7.

The table below shows how one fixed coefficient produces a growing penalty as a token repeats.

Prior occurrencesLogit reductionApproximate probability factor
00.0Unchanged
10.5About 0.61 times
31.5About 0.22 times
63.0About 0.05 times
Effect of a 0.5 coefficient as a token repeats

The probability factor is approximate because softmax renormalizes over all tokens. A penalty on one token redistributes probability mass to every other candidate, so the actual effect depends on how dominant the penalized token was before the adjustment.

Value ranges and typical settings

Most interfaces accept frequency penalty values between -2.0 and 2.0, with a default of 0, which disables the mechanism. Positive values discourage repetition; negative values encourage repetition, which is rarely useful outside special formats such as lists that must reuse fixed field names.

The following table summarizes common ranges and the behavior each one tends to produce in practice.

Value rangeTypical behaviorCommon use
0.0No adjustmentCode, extraction, factual answers
0.1 to 0.5Mild reduction of repeated phrasingGeneral chat and drafting
0.5 to 1.0Noticeably more varied vocabularyBrainstorming, creative writing
Above 1.0Avoidance of common words, degraded grammarRarely justified
Below 0.0Increased repetitionNarrow templated formats
Typical frequency penalty ranges

Exact effects differ between models and implementations. Some runtimes count only tokens in the generated output, others also include prompt tokens, and local inference engines often restrict counting to a sliding window of recent tokens, for example the last 64 tokens. These details change how aggressive a given value feels, so settings rarely transfer unchanged between providers.

Frequency penalty belongs to a family of anti-repetition and diversity controls that differ in what they measure and how they adjust the distribution. The presence penalty applies a flat, one-time reduction once a token has appeared, which pushes the model toward new topics rather than simply new words. The repetition penalty, introduced with the CTRL model, divides positive logits by a factor such as 1.1 to 1.3, making it multiplicative rather than additive.

ControlWhat it measuresType of adjustmentMain effect
Frequency penaltyNumber of prior occurrencesAdditive, scales with countFewer repeated words and loops
Presence penaltyWhether a token has occurredAdditive, flatEncourages new topics
Repetition penaltyWhether a token has occurredMultiplicative on logitsGeneral anti-repetition
TemperatureNothing, applies globallyScales all logitsMore or less randomness
No-repeat n-gram sizeRepeated n-gramsHard blockPrevents exact phrase repeats
How anti-repetition and diversity controls differ

Unlike temperature, frequency penalty is history-dependent: it affects only tokens that have already been generated, so it targets repetition specifically instead of making every choice more random. Unlike a hard n-gram block, it is soft and graded, so a necessary repeated word can still appear if the model rates it highly enough.

Choosing a frequency penalty for a task

The right setting depends on whether repetition is an error or a requirement in the target output. Many tokens must legitimately recur: variable names in code, units in a data table, a product name in a support answer or the keys of a JSON object. Penalizing them can produce inconsistent identifiers and broken structure.

SituationRecommended settingReason
Source code generation0.0Identifiers must repeat exactly
Structured JSON output0.0Keys and syntax recur by design
Long-form article drafting0.2 to 0.5Reduces echoed phrases
Brainstorming idea lists0.5 to 0.8Pushes toward distinct items
Small model stuck in loops0.3 to 0.6Breaks degenerate repetition
Translation and summarization0.0 to 0.2Fidelity matters more than variety
Decision guide for frequency penalty settings

A sound approach is to change one sampling parameter at a time and evaluate the outputs on a fixed set of prompts. Frequency penalty interacts with temperature and top-p, so raising several of them together makes it hard to see which change caused an improvement or a regression.

Limitations and common mistakes

Frequency penalty is a blunt statistical instrument. It operates on token identifiers, not on meaning, so it cannot detect a paraphrased repetition of the same idea and it may penalize harmless subword pieces that happen to recur, such as common suffixes produced by byte-pair encoding.

  • Over-penalizing function words: high values make the model avoid articles and prepositions, which produces stilted or ungrammatical text.
  • Treating it as a quality knob: the penalty does not reduce hallucination or improve accuracy; it only changes word choice.
  • Ignoring tokenization: the same word with and without a leading space can be different tokens, so counts may not match intuition.
  • Copying values across models: modern instruction-tuned models repeat far less than early base models and often need little or no penalty.
  • Penalizing required structure: in code and data formats, any positive value can corrupt names, keys or delimiters.

Some recent reasoning-oriented APIs ignore or reject frequency penalty entirely, because their internal decoding is managed by the provider. Checking the parameter support of each model avoids silent no-ops.

Explained without jargon

When a language model writes, it picks one word piece at a time from a list of likely options. Sometimes it gets stuck and keeps choosing the same words, like a speaker who repeats a favorite phrase in every sentence.

Frequency penalty is a small rule that says: the more often you have used a word, the less attractive it becomes. Each repeat makes that word a little harder to pick again. A gentle setting makes the writing feel fresher, while a very strong setting makes the model avoid even ordinary words and the text starts to sound odd. The key point is that it changes word choice, not knowledge.

A typical use case

A product team uses a mid-sized open-weight model to generate first drafts of help center articles of about 600 words each. Reviewers notice that drafts often repeat the same sentence opener, such as a recurring phrase about simple steps, and occasionally fall into a loop that restates the final paragraph twice.

The team runs a small test on 40 fixed prompts. With temperature at 0.7 and frequency penalty at 0, about one draft in eight contains a looped passage. Raising the frequency penalty to 0.4 while keeping every other setting unchanged removes the loops in the test set and reduces repeated openers, while product names and menu labels still appear correctly. A trial at 1.2 produces awkward synonyms for interface terms, so the team settles on 0.4 for articles and keeps 0 for the separate code-snippet pipeline, where identifiers must repeat exactly.