Tokens per megawatt
Also known as tokens per MW, power-normalized throughput
In plain English
Tokens per megawatt asks how much AI output a data center can produce from a fixed amount of available power.
Technical definition
Tokens per megawatt measures useful inference throughput relative to a data center power budget.
Typical unit
tokens/second per provisioned utility MW
Engineering details
InferenceX uses all-in provisioned utility power, including overhead for power delivery and cooling. Chip thermal design power covers only the accelerator, so it is less useful for facility-level capacity planning.
Why it matters
Power availability is often the binding constraint on new AI deployments. A system that produces more tokens per provisioned megawatt can serve more demand from the same utility allocation even if its individual accelerators draw more power.
How to read it in InferenceX
Compare tokens/MW at the same model, workload shape, precision, and interactivity. Otherwise a high-throughput low-interactivity point can appear efficient while failing the target user experience.
Source material
See the concept in real benchmarks
InferenceMAX: Open Source Inference Benchmarking
NVIDIA GB200 NVL72, AMD MI355X, Throughput Token per GPU, Latency Tok/s/user, Perf per Dollar, Cost per Million Tokens, Tokens per Provisioned Megawatt, DeepSeek R1 670B, GPTOSS 120B, Llama3 70B
DeepSeekV4 1.6T Day 0 to Day 43 Performance Over Time — Huawei, GB300 NVL72, MI355X, B200
Day 0 Inference Performance, InferenceX, 100x performance improvement in 26 Days, Cost per Million Tokens, Huawei 950DT Inference Trace Analysis