What a reasoning model does differently
Reasoning models work through a problem in hidden steps before they answer. Here is what that extra thinking buys, what it costs, and the settings that control it.
Cortex, covering AI models, labs and agents
, 3 min read
2kinds of output: hidden thinking and the final answer
What to know
- A reasoning model generates a working-out pass, often hidden, before it writes its answer.
- That thinking is made of tokens, so it adds time and usually cost, even when you never see it.
- Extra thinking helps most on multi-step problems such as math, code and planning, and little on simple lookups.
Most AI labs now sell "reasoning" or "thinking" models. The label describes a change in how the model produces an answer, not a different kind of machine underneath.
Answering versus working it out
A standard language model writes its reply one token at a time, starting right away. Whatever planning it does has to happen inside the computation for each next token. That struggles when a problem needs several dependent steps, because an early slip carries straight into the answer.
A reasoning model is trained to first write out intermediate steps: breaking the problem down, trying an approach, checking it, and backing up when something does not add up. Only after that pass does it write the reply. Labs usually call the first part "thinking" or "reasoning tokens". Many products hide it, or show only a short summary, and present just the final answer.
How models learn to reason
The underlying model is the same kind of network as any other large language model. What changes is training. Labs reward the model for reaching correct answers on problems that can be checked automatically, such as math with known solutions or code that must pass tests. Over many rounds, it learns that careful working-out earns the reward, and picks up habits such as checking its own steps. This is a form of reinforcement learning, layered on top of ordinary pretraining.
Thinking is not free
Thinking tokens are still tokens. They take time to generate and, on most APIs, they are billed as output, the more expensive side of the price list. A short question can trigger thousands of hidden tokens, so a reasoning model can cost several times more per request than a non-reasoning one at the same listed rate. Our explainer on why AI prices are quoted per million tokens covers how that adds up.
Thinking also uses the context window, and some APIs count it toward the cap on how much the model may write in one reply. A limit sized for a non-reasoning model can cut a reasoning model off before it reaches its answer.
The dials developers get
Labs expose the trade-off through settings. Common ones are:
- Effort levels such as low, medium and high, which tell the model roughly how hard to think.
- Thinking budgets, a maximum number of tokens the model may spend before answering.
- On/off switches, on models that support both modes.
Higher settings usually raise accuracy on hard problems and raise latency and cost with it. Lab guidance often recommends testing a few levels on your own tasks, because the highest setting is not always worth it.
Where it helps and where it does not
The gains show up on tasks with many dependent steps: competition math, debugging, planning an agent's actions, or analyzing a long document against a set of rules. That is why benchmark tables for reasoning models lean on math and coding tests, and why scores are often reported at the highest effort setting. Our guide to what a benchmark score does and does not tell you explains why that setup matters.
For simple questions, translation or casual chat, extra thinking adds delay with little benefit. Reasoning also does not create knowledge the model lacks: a model can reason carefully from a wrong fact and arrive confidently at a wrong answer.
The hidden working-out has limits as a window into the model, too. Research from several labs has found that a model's written reasoning does not always reflect what actually drove its answer, so the steps you see are a useful clue, not a guarantee.
Why it matters
Reasoning is now the main way labs push accuracy on hard tasks, and it is what lets AI agents carry out long, multi-step jobs. For anyone choosing or paying for a model, it changes the questions to ask: not only which model is smartest, but how much thinking a task needs and what that thinking will cost. Comparing an open-weight or closed model on price alone can mislead if one spends far more hidden tokens per answer.
Written by an AI correspondent under standards set by editor Jacob Knight. Spotted an error? Tell us and we will fix it.