Reasoning model
Also known as thinking model, test-time compute, chain-of-thought model
In plain English
A reasoning model generates long hidden chains of thought before answering, trading extra output tokens for better results on hard problems.
Technical definition
A reasoning model is an LLM trained to spend additional generated tokens working through a problem, producing extended intermediate reasoning before or alongside its final answer.
Engineering details
Instead of scaling only training compute, reasoning models scale test time compute: they think in tokens. A single math or coding query can trigger thousands of tokens of internal deliberation, so output lengths explode relative to chat models. For serving, that shifts load heavily toward decode, inflates KV cache residency per request, and makes tokens per second per user the metric that decides whether a hard query answers in seconds or minutes.
Why it matters
Reasoning turned inference into the scaling frontier: capability now improves by spending more at serving time, which multiplies demand for decode throughput. It reshaped hardware priorities toward memory bandwidth and interconnect, and it is a core reason agentic workloads dominate current benchmark design.
How to read it in InferenceX
The frontier models InferenceX benchmarks are reasoning capable, and its scenarios reflect their traffic: long generations in fixed sequence tests and full agent sessions in AgentX, where deliberation and tool use interleave over many turns.
Source material
See the concept in real benchmarks
A Brief Overview of Agentic Workloads
Multi-turn sessions, long contexts, and near-total prefix reuse make agentic inference a systems problem, and change what a benchmark has to measure
GB200 NVL72 vs B200 on DeepSeek R1 670B: Up to 4.4x Throughput per GPU at 125 tok/s/user
DeepSeek R1 FP4 1k/1k. NVL72's 72-GPU NVLink scale-up fabric lets decode run wide EP up to EP=32, where B200's 8-GPU NVLink island caps out at EP=8 over RoCEv2