Thermal design power
Also known as TDP, board power, all-in power
In plain English
TDP is the sustained power a chip is designed to draw and shed as heat, the headline wattage on every accelerator spec sheet.
Technical definition
Thermal design power is the maximum sustained power envelope a chip is engineered to operate within, which its cooling system must dissipate continuously.
Engineering details
Modern accelerators run around or above the kilowatt mark per chip, and a full rack system multiplies that into six figures of watts. TDP alone also undersells the true bill: memory, networking, CPUs, power conversion losses, and cooling overhead stack on top, which is why all in power per chip is meaningfully higher than the chip TDP. Datacenter capacity is sold in megawatts, so these envelopes translate directly into how many accelerators a site can host.
Why it matters
Power has become the binding constraint of AI buildout, ahead of capital in many markets. Rising per chip TDP forced the shift to liquid cooling and made performance per watt, not just performance per dollar, a primary axis for comparing silicon generations.
How to read it in InferenceX
InferenceX derives energy per token and tokens per megawatt using per chip all in power figures that include cooling and infrastructure overhead above TDP, and the PowerX workstream is extending this from rated figures toward measured draw during runs.
Source material
See the concept in real benchmarks
InferenceX v2: NVIDIA Blackwell Vs AMD vs Hopper - Formerly InferenceMAX
GB300 NVL72, MI355X, B200, H100, Disaggregated Serving, Wide Expert Parallelism, Large Mixture of Experts, SGLang, vLLM, TRTLLM
Vera Rubin NVL72 vs GB200 NVL72? Inference TCO & Architecture Analysis
Rubin LUT Based Tensor Core, Feynman, Rack Scale, Perf Per MegaWatt, Perf Per Dollar, Software Improvements, Public Rubin Software, PyTorch, vLLM, OpenAI Triton