Temperature is a sampling parameter that controls how random the output of a language model is by rescaling the model’s scores for possible next tokens before one is chosen. In language models, a low temperature makes generation more deterministic and repetitive, while a high temperature makes it more varied and more likely to pick unusual tokens.
Temperature does not change what the model knows or how it was trained. It only changes how the model’s probability distribution is turned into a concrete choice at each step of generation, which makes it one of the most commonly adjusted inference settings alongside top-p, top-k and maximum output length.
How temperature reshapes token probabilities
At each generation step, an LLM produces a logit, a raw score, for every token in its vocabulary. The softmax function converts these logits into probabilities. Temperature, written T, divides every logit by T before the softmax, so the probability of token i becomes proportional to exp(logit_i / T).
Dividing by a value below 1 stretches the differences between logits, concentrating probability on the top tokens. Dividing by a value above 1 compresses the differences, flattening the distribution so that lower-ranked tokens gain probability. At T = 1 the model’s original distribution is used unchanged.
| Temperature | Distribution shape | Top token probability in the example | Typical behavior |
|---|---|---|---|
| Close to 0 | Almost all mass on one token | Near 100 percent | Greedy, repeatable output |
| 0.5 | Sharpened | About 78 percent | Focused, minor variation |
| 1.0 | Original model distribution | About 60 percent | Natural variety |
| 1.5 | Flattened | About 52 percent | Creative, less coherent |
| 3.0 | Close to uniform over plausible tokens | About 43 percent | Frequent errors and drift |
The example assumes three candidate tokens with probabilities of 60, 30 and 10 percent at T = 1; real distributions span thousands of tokens and change at every step. A temperature of exactly zero is undefined mathematically, so systems treat it as greedy decoding, always picking the highest-scoring token.
Typical temperature ranges and settings
Most interfaces accept temperatures between 0 and 1 or 0 and 2, and the useful range for most tasks is narrower. The best value depends on whether the task has one correct answer or rewards variety.
| Task | Suggested range | Reason |
|---|---|---|
| Data extraction and classification | 0 to 0.2 | Consistent, repeatable labels |
| Code generation | 0 to 0.4 | Syntax and logic tolerate little randomness |
| Factual question answering | 0 to 0.5 | Reduces unlikely, unsupported tokens |
| Conversational assistants | 0.5 to 0.9 | Natural phrasing without drift |
| Brainstorming and creative writing | 0.8 to 1.2 | Wider exploration of ideas |
| Generating diverse candidates | 1.0 or higher | Varied samples for later ranking |
These ranges are starting points, not rules. Models differ in how calibrated their raw distributions are, so the same temperature can feel conservative in one model and adventurous in another. Testing on representative prompts is the reliable way to choose.
Temperature compared with other sampling parameters
Temperature reshapes the whole distribution, whereas truncation methods such as top-k and top-p remove unlikely tokens before sampling. They are often combined, and understanding the difference prevents conflicting settings.
| Parameter | What it does | Typical value |
|---|---|---|
| Temperature | Sharpens or flattens all probabilities | 0 to 1.2 |
| Top-k sampling | Keeps only the k most likely tokens | 20 to 100 |
| Top-p (nucleus) sampling | Keeps the smallest set reaching probability p | 0.9 to 0.95 |
| Min-p sampling | Drops tokens below a fraction of the top probability | 0.05 to 0.1 |
| Repetition and frequency penalties | Lower scores of already used tokens | Small positive values |
| Greedy decoding | Always picks the top token | Equivalent to temperature 0 |
A common recommendation is to adjust temperature or top-p, but not both aggressively. Raising temperature while also restricting top-p partly cancels the effect, and the resulting behavior becomes hard to predict.
Temperature, reliability and hallucination
Higher temperatures increase the chance of selecting low-probability tokens, and one unlikely token can steer the rest of a response off course, because each new token becomes part of the context for the next. This is one reason high temperature correlates with more factual errors and hallucination.
The reverse does not hold, however. Temperature 0 does not make a model truthful: if the model’s most probable continuation is wrong, greedy decoding will produce that wrong answer confidently and consistently. Hallucination originates in the model and its prompt, while temperature only influences how often less likely paths are explored.
Low temperature also has costs. Near-greedy decoding is prone to repetitive loops in long outputs and produces bland, formulaic text, which is why open-ended generation usually works better at moderate values.
For dependable factual output, grounding the prompt in retrieved documents and requesting structured answers reduce errors far more than temperature tuning alone. Temperature is best treated as a fine adjustment of style and diversity, not as a correctness control.
Common mistakes with temperature
- Expecting perfect determinism at temperature 0: parallel hardware arithmetic and batching can still cause small run-to-run differences.
- Raising temperature to fix weak or vague prompts, when clearer instructions or examples would help more.
- Using one global value for every task in an application, instead of tuning per task type.
- Comparing models at different temperatures, which makes evaluation results misleading.
- Assuming reasoning-focused models accept the setting; some fix temperature internally and ignore or reject custom values.
Temperature also interacts with sampling multiple answers. Techniques such as self-consistency generate several responses at a moderate temperature and select the most common final answer, which can improve accuracy on reasoning tasks precisely because of controlled randomness.
In everyday language
When a language model writes, it chooses each next word from a list of options, each with a likelihood. Temperature is like a dial for adventurousness. Turned down, the model almost always picks the safest, most obvious word. Turned up, it is more willing to pick surprising words.
Neither setting is better in general. Low temperature suits tasks with one right answer, such as filling in a form, while higher temperature suits tasks where fresh ideas matter, such as suggesting names for a new product.
Seen in the real world
A software company builds two features on the same LLM. The first reads incoming support tickets and outputs a category and priority in a fixed JSON format. The team sets temperature to 0, so identical tickets receive identical labels, and measures labeling consistency across repeated runs as part of its tests.
The second feature drafts five alternative subject lines for marketing emails. At temperature 0, the five drafts were nearly identical, so the team raises it to 0.9 with top-p at 0.95. The suggestions become clearly varied while staying on topic, and the marketing staff pick one or edit it. Choosing the temperature per task, rather than one value for the whole product, gives reliable classification and useful creativity from the same model.