Logits are the raw, unnormalized scores that a neural network produces in its final layer, one score per possible output class. In language models, the classes are the entries of the vocabulary, so at every generation step the model emits one logit per token, often more than 100,000 numbers, and these scores decide which token is likely to come next.
Logits sit at the boundary between the network and the decoding strategy. They are not probabilities: they can be negative, they do not sum to one, and only their relative differences carry meaning. The softmax function converts them into a probability distribution, and nearly every generation control in artificial intelligence text systems, from temperature to repetition penalty, works by adjusting logits before that conversion.
How a language model produces logits
A model produces logits by projecting its final hidden state onto the vocabulary. In a decoder-only transformer, the last layer outputs a vector of dimension d, typically 2,048 to 16,384, for each position. A matrix called the language modeling head (or unembedding matrix) multiplies that vector and yields a vector of length V, the vocabulary size.
Each entry of the result is the dot product between the hidden state and the learned vector for one token. A large positive value means the hidden state points in the same direction as that token, so the model rates it as a plausible continuation. Many models tie the output head to the input embedding matrix, which saves parameters and makes the geometry of logits mirror the geometry of token embeddings.
The table below traces the path from hidden state to sampled token in a single generation step.
| Stage | Operation | Output |
|---|---|---|
| Hidden state | Final transformer layer and normalization | Vector of size d |
| Projection | Multiply by the unembedding matrix | Logit vector of size V |
| Adjustment | Penalties, bias, temperature, masking | Modified logits |
| Normalization | Softmax over all entries | Probability distribution |
| Selection | Greedy choice or sampling | One token ID |
Logits, probabilities and logprobs
The softmax function maps logits to probabilities by exponentiating each logit and dividing by the sum of all exponentials. Because of the exponent, a gap of one unit between two logits means one token is about 2.7 times as likely as the other, and a gap of five units means roughly 148 times. Adding the same constant to every logit changes nothing, which is why the absolute values are arbitrary.
Developers often confuse logits with log-probabilities, the values many APIs expose as logprobs. A logprob is the logarithm of the final probability, so it is always zero or negative and already normalized. Logits are the pre-normalization scores; subtracting the log of the softmax denominator from a logit gives its logprob.
| Quantity | Range | Sums to one | Typical use |
|---|---|---|---|
| Logit | Any real number | No | Internal scoring and decoding controls |
| Probability | 0 to 1 | Yes | Sampling and calibration |
| Logprob | Negative infinity to 0 | Yes, after exponentiation | Scoring, perplexity, confidence signals |
The name comes from statistics, where the logit function is the log-odds of a probability. In a two-class model with a sigmoid output, the single output really is a log-odds value; in a multiclass softmax model, the word is used more loosely for any unnormalized score.
Decoding controls that operate on logits
Most text generation settings are transformations of the logit vector applied before softmax. Understanding this makes their interaction predictable: each control either rescales, shifts or masks logits, and the order in which an inference engine applies them affects the result.
| Control | Effect on logits | Practical result |
|---|---|---|
| Temperature | Divides all logits by T | Below 1 sharpens, above 1 flattens the distribution |
| Top-k sampling | Sets all but the k highest to negative infinity | Restricts choice to k candidates |
| Top-p sampling | Masks tokens outside the smallest set reaching mass p | Adaptive candidate pool |
| Repetition penalty | Divides or multiplies logits of seen tokens | Discourages loops |
| Logit bias | Adds a fixed value to chosen token IDs | Forces or bans specific tokens |
| Constrained decoding | Masks tokens that break a grammar | Guarantees valid JSON or schema output |
Greedy decoding simply takes the argmax of the logits, so temperature has no effect on it. Min-p sampling keeps tokens whose probability exceeds a fraction of the top token’s probability, which is equivalent to a threshold relative to the highest logit. Frequency and presence penalties subtract values proportional to how often a token has already appeared.
Where logits matter in practice
Logits are the main interface for inspecting and steering a model without retraining it. Teams use them in classification, evaluation, safety filtering and model compression, each relying on a different property of the scores.
| Use case | How logits are used | Key caution |
|---|---|---|
| Zero-shot classification | Compare logits of label tokens such as Yes and No | Label tokenization must be checked |
| Knowledge distillation | Train a student to match teacher logits | Needs access to full logits, not text |
| Confidence estimation | Measure the margin between top logits | Raw margins are often miscalibrated |
| Structured output | Mask invalid tokens at each step | Masking can distort the distribution |
| Interpretability | Project intermediate layers through the head | Early-layer readouts are approximate |
The logit lens technique applies the unembedding matrix to hidden states from intermediate layers, showing how the prediction forms layer by layer. Speculative decoding compares draft-model and target-model distributions derived from logits to accept or reject proposed tokens without changing the output distribution.
Limitations and common misconceptions
A high logit is not a measure of factual truth. It reflects how well a continuation fits patterns learned in training, so a model can assign a confident score to a hallucination. Instruction tuning and preference training often make output distributions sharper and less calibrated than those of the base model.
- Logits are not comparable across models: different vocabularies, scales and training make raw values meaningless outside one model.
- Single-token views mislead: a word split into several tokens needs the product of several probabilities, not one logit.
- Numerical precision matters: large vocabularies in 16-bit formats risk overflow, so softmax is computed in 32-bit precision with the maximum subtracted first.
- Access is often restricted: many hosted APIs expose only top logprobs, partly to prevent distillation of proprietary models.
Some models apply logit soft-capping, a tanh-based squashing that keeps scores within a fixed range such as plus or minus 30, which stabilizes training but changes how temperature affects sampling.
Put simply
Before a language model writes its next word, it gives a score to every word it knows. Those scores are the logits. A higher score means the word fits better, and a lower one means it fits worse. The scores are not percentages yet; a separate step turns them into odds that add up to one hundred percent.
Think of judges at a competition who write down raw points for every contestant. The points only mean something compared with each other. Settings such as temperature change those points before the winner is picked, which is how the same model can sound careful in one app and more creative in another.
In practice
A support team uses a self-hosted open-weight model to route incoming tickets into four categories: billing, technical, account and other. Instead of asking the model to write an answer and parsing free text, the engineers read the logits of the four label tokens at the position where the category should appear and apply softmax only over those four values.
This approach makes every response valid by construction and produces a usable confidence score. The team sends any ticket whose top category falls below 0.6 probability to a human agent. After calibrating the threshold on 2,000 labeled tickets, they find the automated routes match human labels in about 94 percent of cases, while roughly one ticket in eight goes to manual review. Reading logits directly replaced a fragile text parser with a single, measurable decision rule.