Large Language Models · entry 08/11 · 3 min read

Reasoning models

Models trained to think before answering trade serving compute for correctness — a second scaling axis, a new cost curve, and a transcript you should not take at face value.

A second dial

For a decade, better meant bigger: more parameters, more data, more training compute. Reasoning models added an orthogonal dial — let the model spend more compute at answer time. Given a hard problem it emits a long internal chain of reasoning, exploring, checking and backtracking before committing to an answer, and accuracy on math, code and multi-step logic rises with the length of that chain. The knob is real and it is spendable per request: the same weights, thinking longer, do measurably better work.

Trained, not prompted

This isn't chain-of-thought prompting with a bigger budget — the behavior is trained in. The frontier recipe is a supervised warm-up on long reasoning traces, then reinforcement learning against verifiable answers: sample many attempts at problems with checkable solutions, reward the ones that actually pass, and the habits that produce passing answers — decomposing, trying an approach, noticing it fails, starting over — get reinforced. What emerges looks like deliberation because deliberation is what survived the filter, though how much of it the base model could already do — and RL merely made reliable — is genuinely contested. Those traces also distil cheaply, which is how most small reasoning models are actually made. It explains the capability's shape too: strongest where answers can be checked, softest where they cannot.

The economics invert

Reasoning tokens are billed like any others, and a long chain can cost more than the visible answer by an order of magnitude — the decode phase that dominates serving cost, deliberately extended. Latency follows: seconds to minutes rather than under a second. So routing becomes the design question. Extraction, classification, and formatting want the fast path; a gnarly migration plan or a proof wants the slow one. Most production systems now run a mix, and the interesting engineering is deciding which requests deserve the expensive brain — often by trying cheap first and escalating on a failed check.

Where it doesn't help

Thinking longer cannot supply facts the model does not have — retrieval still owns that job — and it does not fix a task that was underspecified. On simple requests extra reasoning can even hurt, talking a correct instinct out of itself. Treat the reasoning budget as a resource with a return curve that flattens: measure where yours flattens rather than paying for maximum effort by default.

Failure mode

Reading the chain as a full account of the reason. The trace is doing real work — those tokens are the serial computation, which is why cutting them short costs accuracy — but it is not a faithful report of everything driving the answer: models demonstrably lean on cues and hints they never verbalize, and a well-structured chain can accompany a wrong answer. Providers summarize or hide raw traces mostly to prevent distillation, and deliberately leave the raw chain un-optimized so it stays readable as a monitoring signal. Treat it as a fragile signal rather than an audit trail — useful for debugging your prompt and spotting missing context — and grade the answer, as always.