Min-p sampling

On this page 7

Min-p sampling is a truncation method for text generation in language models that keeps only the candidate tokens whose probability is at least a fixed fraction of the top token’s probability, then samples from that reduced set. Because the cutoff scales with the model’s confidence, it removes the unlikely tail aggressively when one continuation clearly dominates and lets many options through when the distribution is flat.

In artificial intelligence practice, min-p is used as an alternative or complement to top-p and top-k sampling, especially when developers want higher temperature for creativity without the incoherent output that a long tail of improbable tokens tends to produce. It is a sampling-time setting: it changes how the next token is chosen, not what the model has learned.

How min-p sampling filters candidate tokens

Min-p sampling works in four steps applied at every generated token. The model first produces logits for the whole vocabulary, which are converted with softmax into a probability distribution. The sampler then finds the maximum probability p_max among all tokens and multiplies it by the user parameter min_p, called the base threshold.

Every token whose probability falls below min_p × p_max is discarded. The surviving probabilities are renormalized to sum to one, and the next token is drawn at random from them. The procedure repeats for each position, so the size of the candidate pool changes continuously with the context.

The following table shows how the same setting of min_p = 0.1 produces very different cutoffs depending on the shape of the distribution.

SituationTop token probabilityCutoffTypical tokens kept
Confident step0.900.091 to 2
Moderately sure step0.400.045 to 15
Open-ended step0.080.008Dozens to hundreds
Effect of min_p = 0.1 on different distributions

This dynamic threshold is the defining property of the method. A factual completion such as a date or a code keyword collapses to almost deterministic output, while a creative position such as the next adjective in a story keeps a wide, varied pool.

Choosing the min_p value

The base threshold is usually set between 0.01 and 0.2, and most practical configurations sit in the range 0.05 to 0.1. Lower values admit more of the tail and increase diversity; higher values prune harder and push output toward the most probable phrasing. The right value also depends on temperature, because temperature reshapes the distribution that the threshold is applied to.

Min_p valueBehaviorTypical use
0.01 to 0.03Very permissive, high diversityBrainstorming, poetry at moderate temperature
0.05 to 0.1Balanced coherence and varietyGeneral chat, fiction, summaries
0.15 to 0.3Strong pruning, conservativeStructured text, high-temperature runs
0 (disabled)No min-p filteringPure temperature or other truncation
Common min_p settings and their effects

Because the filter adapts to confidence, min-p tolerates temperatures of 1.5 or higher better than top-p does. At such temperatures the distribution flattens and the tail grows, yet the relative cutoff still removes tokens that remain far below the leader.

Min-p compared with other truncation methods

Min-p belongs to the family of truncation samplers, which all remove part of the distribution before sampling. They differ in what defines the boundary: a fixed count, a cumulative mass or a ratio to the top token. The comparison below places min-p alongside its neighbors in the decoding strategy toolbox.

MethodCutoff ruleAdapts to confidenceMain weakness
Top-k samplingKeep the k most likely tokensNoSame k for flat and peaked steps
Top-p samplingKeep tokens until cumulative mass reaches pPartlyLong tail admitted when distribution is flat
Min-p samplingKeep tokens at or above min_p × p_maxYesLess intuitive parameter for newcomers
Greedy decodingKeep only the top tokenNot applicableRepetitive, no diversity
Truncation methods in language model decoding

The key contrast with top-p is how each method behaves on a flat distribution. Top-p with p = 0.9 must collect tokens until 90 percent of the mass is covered, which can mean hundreds of weak candidates. Min-p looks only at the ratio to the strongest token, so it ignores the cumulative tail and keeps candidates that are individually plausible.

Configuring min-p in practice

Min-p is exposed as a min_p parameter in many open-source inference engines and libraries, including llama.cpp, vLLM and Hugging Face Transformers, and in several local-model interfaces. Hosted APIs of closed models often do not offer it, so its use is most common with open-weight models run locally or on self-managed servers.

  • Disable overlapping filters when testing: set top-p to 1.0 and top-k to 0 so that the effect of min-p is isolated.
  • Check the sampler order in the chosen engine, because applying temperature before or after min-p changes which tokens survive.
  • Tune with temperature together, adjusting min_p upward when temperature rises above 1.0.
  • Keep repetition penalty moderate, since aggressive penalties can push the top token down and distort the ratio.

The decision table below summarizes starting configurations that practitioners commonly adjust from.

TaskTemperatureMin_pReason
Creative fiction1.0 to 1.50.05 to 0.1Variety without broken sentences
Conversational assistant0.7 to 1.00.05Natural tone with stable facts
Code generation0.2 or lower0.1 or higherPrecision outweighs diversity
Data extraction0Not neededGreedy output is already deterministic
Starting configurations by task

Limitations and common misconceptions

Min-p is a decoding heuristic, not a quality guarantee. It cannot correct a model that assigns high probability to a wrong fact, so it does not prevent hallucination. It only reduces the chance that a low-probability token derails an otherwise sound continuation.

  • Misconception: higher min_p means better text. Excessive values make output bland and repetitive, approaching greedy decoding.
  • Misconception: min-p replaces temperature. Temperature controls the shape of the distribution; min-p only trims it.
  • Limitation: evaluation is contested. Reported gains in creativity and coherence come from specific benchmarks and human ratings, and results vary by model and task.
  • Limitation: stacking filters hides effects. Combining top-k, top-p and min-p makes it hard to see which filter actually determined the output.

A practical failure mode appears with heavily penalized distributions. When repetition or frequency penalties lower the leading token, p_max shrinks and the threshold drops with it, so the filter becomes unexpectedly permissive at exactly the steps where the model is uncertain.

The idea in brief

When a language model writes, it guesses the next word from a ranked list of options, each with its own likelihood. Min-p sampling throws out every option far weaker than the favorite. If the favorite is very strong, only one or two options stay; if several words are almost equally good, many stay.

Think of a talent show judge who only invites back acts that scored at least one tenth of the winner’s score. In a weak round many acts return, in a round with a clear star only a few do. The result is text that can be lively without becoming nonsense.

A practical scenario

A small game studio runs an open-weight model on its own server to generate dialogue for background characters. With top-p at 0.95 and temperature 1.3, the team finds that roughly one line in twenty contains an odd word or a broken phrase, which writers must fix by hand.

The developers switch to min_p = 0.08 at temperature 1.3, with top-p set to 1.0 and top-k disabled. Lines remain varied, characters still sound different from one another, and the odd words largely disappear because tokens far below the leading candidate never enter the pool. After reviewing several hundred lines, the writers report that manual corrections dropped substantially, and the studio keeps the setting as its default for ambient dialogue while using lower temperature for quest instructions that must stay precise.