Logprobs

Also known as: log probabilities

On this page 7

Logprobs, short for log probabilities, are the natural logarithms of the probabilities that a model assigns to each token it could produce at a given position. In language models, every generation step ends with a probability distribution over the whole vocabulary, and logprobs expose that distribution, or its most likely entries, as numbers that are always zero or negative.

Many inference APIs and open-source runtimes can return the logprob of each sampled token together with the top alternative tokens at the same step. This makes logprobs one of the few direct windows into model behavior in applied artificial intelligence: they support confidence estimation, classification, answer ranking, evaluation and the detection of uncertain or possibly hallucinated passages.

How logprobs are computed from logits

Logprobs are derived from the logits, the raw unnormalized scores that the final layer of a decoder-only model produces for every token in the vocabulary. A softmax function converts those scores into probabilities that sum to one, and the logarithm of each probability is its logprob. In practice, runtimes compute the log-softmax directly, because it is numerically stable and avoids underflow for very unlikely tokens.

Because a probability lies between 0 and 1, its natural logarithm lies between negative infinity and 0. A logprob of 0 means the model was certain, a logprob of about -0.69 corresponds to 50 percent, and a logprob of -4.6 corresponds to roughly 1 percent. Converting back is simple: the probability equals e raised to the logprob.

The table below translates typical logprob values into probabilities and the way practitioners usually read them.

LogprobProbabilityUsual interpretation
0.00100%Fully determined token, such as a closing bracket
-0.05About 95%Highly confident choice
-0.6950%Two roughly equal candidates
-2.3010%Plausible but unlikely alternative
-4.601%Rare token, often a sign of uncertainty
-9.210.01%Almost excluded by the model
Reading individual token logprobs

Why logarithms instead of raw probabilities

The logarithmic form exists mainly because sequence probabilities multiply. An autoregressive model assigns a sentence the product of the conditional probabilities of its tokens, and a product of hundreds of small numbers quickly falls below the precision of floating-point arithmetic. Logarithms turn that product into a sum of logprobs, which stays in a manageable range and is cheap to compute.

The same sums connect logprobs to the standard training and evaluation metrics of language modeling. The negative log-likelihood of a text is the negated sum of its token logprobs, the cross-entropy loss is its average per token, and perplexity is the exponential of that average. A lower perplexity therefore means the model assigned higher logprobs to the observed text.

These related quantities are often confused, so the table below separates them.

QuantityDefinitionTypical use
LogitRaw score before softmaxSampling, constrained decoding, bias adjustments
ProbabilitySoftmax of the logitsHuman-readable confidence
LogprobNatural log of the probabilityScoring tokens and sequences
Sequence logprobSum of token logprobsRanking complete answers
PerplexityExponential of mean negative logprobComparing models on a corpus
Logprobs and related quantities

What logprobs reveal and how they are used

The most common use of logprobs is token-level confidence analysis. When a model writes a date, a name or a number with a logprob close to zero, the choice was stable; when the chosen token has a low logprob and several alternatives are close, the passage deserves review. Teams use this signal to flag low-confidence spans in generated text before it reaches users.

Logprobs also make language models usable as classifiers without extra training. If a prompt asks for a single label such as yes or no, the logprobs of the candidate label tokens give a calibrated-looking score for each class, which supports thresholds, ranking and the measurement of precision and recall. The table below summarizes the main application patterns.

ApplicationWhat is measuredBenefit
ClassificationLogprobs of label tokensScores and thresholds instead of plain text
Answer rerankingSequence logprob of each candidateSelects the most likely completion
Hallucination screeningLow-logprob tokens in factsHighlights spans for human review
AutocompleteTop alternatives per positionOffers ranked suggestions
Model evaluationLogprob of reference answersMultiple-choice benchmarks without sampling
Data filteringPerplexity of documentsRemoves noisy or garbled training text
Common applications of logprobs

Evaluation harnesses rely heavily on this pattern: for a multiple-choice question, they compute the logprob of each answer option given the question and pick the highest, often after length normalization so that longer options are not penalized merely for containing more tokens.

How sampling settings affect reported logprobs

Logprobs describe the model distribution, but the decoding strategy decides which token is actually emitted. Settings such as temperature, top-p sampling and top-k sampling reshape or truncate the distribution, and implementations differ in whether the returned logprobs reflect the raw distribution or the adjusted one. Engineers should check this detail before comparing values across systems.

With greedy decoding, the emitted token is always the one with the highest logprob. With sampling, a token with a logprob of -3 can still be chosen, which is why a low logprob is not an error, only evidence that the model considered the choice unlikely. The decision table below lists typical configurations.

GoalRecommended settingReason
Stable confidence scoresTemperature 0 or greedy decodingRemoves sampling noise from the chosen tokens
Label classificationOne output token, top 5 to 20 alternativesCaptures every candidate label
Comparing two modelsSame tokenizer and scoring textLogprobs depend on tokenization
Studying diversitySampling with logprobs of sampled tokensShows how often unlikely tokens appear
Configuring generation for logprob analysis

Limitations and common misconceptions

Logprobs are informative but frequently overinterpreted. The most important limitation is that a high logprob measures model confidence, not truth. A model can assign a probability above 90 percent to a false statement it absorbed during training, and instruction tuning and reinforcement learning from human feedback are known to make many chat models less well calibrated than their base models.

  • Tokenization effects: one word may be split into several tokens, so a word-level confidence requires summing or averaging several logprobs.
  • Surface-form competition: probability mass spreads across synonyms and spellings, so a correct answer can receive a modest logprob simply because it has many valid phrasings.
  • Length bias: raw sequence logprobs favor shorter outputs, which is why ranking usually requires normalization.
  • Limited availability: some APIs return only the top few alternatives, restrict logprobs for certain models or omit them for hidden reasoning tokens.
  • Not comparable across vocabularies: values from models with different tokenizers cannot be compared token by token.

Used with these caveats in mind, logprobs remain a cheap, model-native uncertainty signal that complements, but never replaces, external verification such as retrieval or human review.

The simple explanation

Each time a language model writes a word, it first rates every possible next word by how likely it seems. Logprobs are those ratings, written on a special scale. A value near zero means the model felt sure, and a large negative value means it hesitated.

Think of a quiz contestant who says each answer aloud and also whispers how confident they are. The whisper does not prove the answer is right, but it tells you which answers are worth double-checking. Logprobs play the role of that whisper for software that uses language models.

How it is used in practice

A support team at a mid-sized software company routes about 4,000 incoming emails a week into six categories. Instead of asking a language model to write the category name freely, the engineers ask it to answer with a single label token and request the top 10 logprobs at that position, using temperature 0.

For each email, the system converts the label logprobs into probabilities. When the best label exceeds 90 percent probability, the ticket is routed automatically; when it falls below that threshold, or when two labels are within a few points of each other, the email goes to a human queue. After a month of tuning, about 85 percent of tickets are routed automatically, misrouting drops noticeably, and the team uses the stored logprobs to find category definitions that the model regularly confuses.