Reasoning tokens, also called thinking tokens, are the intermediate tokens a model generates before its final answer, used to work through a problem step by step. Language models trained as reasoning models produce this internal scratch work, and in artificial intelligence practice it is treated as a separate category of output: it is billed, counted and limited like other generated text, but it is often hidden or summarized rather than shown to the user.
The concept matters because reasoning tokens turn extra computation into better answers. Spending more of them is the most common form of test-time compute, and controlling how many a model may spend is a central lever for cost, latency and accuracy.
How reasoning tokens are produced
Reasoning tokens are produced by the same autoregressive next-token prediction that generates any other output, but inside a dedicated thinking phase. The model receives the prompt, opens a reasoning segment marked by special delimiter tokens defined in its chat template, writes intermediate steps, closes the segment and then writes the answer.
The behavior is learned rather than scripted. Reasoning models are typically trained with reinforcement learning on verifiable tasks such as mathematics and code, where a correct final answer is rewarded. Over training, the model learns that longer, self-checking reasoning raises the reward, so it starts to explore alternatives, test partial results and correct its own mistakes inside the thinking segment.
The following table separates the token categories that appear in a single request to a reasoning model.
| Token category | Produced by | Visible to user | Billed as |
|---|---|---|---|
| Input tokens | User and system prompt | Yes, as sent | Input |
| Reasoning tokens | Model, thinking phase | Hidden, summarized or shown | Output |
| Answer tokens | Model, answer phase | Yes | Output |
| Cached input tokens | Reused prompt prefix | Yes, as sent | Discounted input |
A key consequence is that reasoning tokens consume the output budget. If a request sets a low max tokens limit, the model may spend it entirely on thinking and return an empty or truncated answer.
Visibility and handling across providers
Providers handle reasoning tokens in three main ways: they hide them completely, return a condensed summary, or expose the full trace. The choice reflects a trade-off between transparency, safety review and protection of model behavior.
| Visibility model | What the user receives | Typical motivation |
|---|---|---|
| Fully hidden | Only a token count | Protects raw traces, reduces misuse |
| Summarized | A shorter description of the reasoning | Gives insight without the full trace |
| Fully visible | The complete thinking text | Common in open-weight models |
| Encrypted pass-through | An opaque block to send back | Preserves reasoning across turns |
In multi-turn conversations, reasoning tokens from earlier turns are usually dropped from the context window or replaced by an encrypted block. This keeps long sessions affordable, but it also means a model does not automatically remember its earlier reasoning unless the interface passes it back. In tool-use loops, many interfaces require the reasoning block to be returned unchanged so that the model can continue the same line of thought after a tool result.
Controlling the reasoning budget
The amount of reasoning is controlled through a thinking budget or an effort setting. A budget sets an approximate ceiling in tokens, while an effort level such as low, medium or high lets the model choose its own length within a band. Some models also use adaptive thinking, deciding per request whether deep reasoning is needed at all.
Budgets are soft targets rather than exact counts. A model may stop well below the ceiling on an easy question and may approach it on a hard one. Simple questions often need a few hundred tokens, while difficult mathematics, planning or coding tasks can consume tens of thousands of tokens.
| Task type | Suggested setting | Reason |
|---|---|---|
| Classification or extraction | Off or minimal | Little benefit, added latency |
| Short factual question | Low | Retrieval matters more than reasoning |
| Multi-step analysis | Medium | Gains from checking intermediate steps |
| Competition math or complex code | High | Accuracy scales with thinking length |
| Agentic tool workflows | Medium, interleaved | Reasoning between tool calls |
Accuracy usually rises with budget on hard tasks, but the gain is roughly logarithmic: each doubling of reasoning tokens buys a smaller improvement. Past a point, extra thinking yields diminishing or even negative returns, a pattern sometimes called overthinking.
Reasoning tokens compared with related ideas
Reasoning tokens are related to chain-of-thought prompting but are not the same thing. Chain-of-thought is a prompting technique that asks any model to show its steps in the answer, whereas reasoning tokens come from a model trained to think in a separate phase without being asked.
| Concept | Where the steps appear | Requires special training |
|---|---|---|
| Reasoning tokens | Separate thinking segment | Yes |
| Chain-of-thought prompting | Inside the visible answer | No |
| Test-time compute | Umbrella term for extra inference work | Depends on method |
| Beam search | Parallel candidate sequences | No |
The question of chain-of-thought faithfulness applies directly. A reasoning trace is not guaranteed to reflect the computation that actually produced the answer, so a plausible trace is evidence, not proof, of how the model decided.
Costs, limits and common mistakes
The main practical risk is underestimating cost and latency. Because reasoning tokens are billed at the output rate and are often invisible, a response that looks short can cost many times more than a comparable answer from a non-reasoning model.
- Setting max tokens too low: the thinking phase exhausts the limit and the answer is cut off.
- Ignoring hidden usage: cost dashboards must read the reasoning count from usage metadata, not from the visible text length.
- Using high effort everywhere: simple tasks become slower and more expensive with no measurable gain.
- Treating the summary as the full trace: a summary omits steps and cannot be audited as complete.
- Prompting for step-by-step output on top of thinking: this duplicates work and can lower quality.
Latency also grows with reasoning length, since every token is generated sequentially. For interactive interfaces, streaming the summarized reasoning or a progress indicator helps users tolerate the delay.
A non-technical view
Reasoning tokens are the notes a model writes to itself before it answers. Think of a student solving a hard math problem on scrap paper and then copying only the final result onto the exam sheet. The scrap paper is the reasoning, and the clean line on the exam is the answer.
Those notes take time and money even when nobody reads them. More notes usually mean better answers on hard questions, but on easy questions they only make the reply slower and more expensive.
A concrete example
A software team uses a reasoning model to review pull requests for concurrency bugs. Their first setup used high effort on every request with a 4,000-token output limit. On large diffs, the model often spent the entire limit thinking and returned no review, while small formatting changes still triggered several thousand reasoning tokens.
The team raised the output limit to 32,000 tokens, switched formatting-only changes to low effort, and kept high effort for files touching locks and shared state. They also logged reasoning token counts from the usage metadata. After the change, empty responses disappeared, average cost per review fell by more than half, and the model still caught the race conditions that mattered.