Power usage effectiveness
Also known as PUE, datacenter efficiency ratio
In plain English
PUE measures how much total datacenter power is consumed for every watt that actually reaches the computing equipment inside it.
Technical definition
Power usage effectiveness is the ratio of total facility power to IT equipment power, where a value of 1.0 would mean every watt goes to computing.
Engineering details
Cooling, power conversion, and facility systems consume energy on top of the servers themselves. A PUE of 1.5 means half again as much power is spent on overhead as on IT load, while modern hyperscale AI facilities push toward 1.1 and below with liquid cooling and high efficiency distribution. Because AI campuses are sized in hundreds of megawatts, small PUE differences move enormous absolute energy and cost figures.
Why it matters
PUE links chip level efficiency to facility economics: every watt a chip draws is multiplied by PUE at the utility meter. As power availability gates AI buildout, facility efficiency became part of the competitive calculus alongside silicon and software.
How to read it in InferenceX
The all in power figures behind InferenceX energy per token and tokens per megawatt metrics incorporate facility overhead consistent with modern AI datacenter PUE, so its cost and energy comparisons reflect delivered facility economics rather than bare chip wattage.
Source material
See the concept in real benchmarks
InferenceX v2: NVIDIA Blackwell Vs AMD vs Hopper - Formerly InferenceMAX
GB300 NVL72, MI355X, B200, H100, Disaggregated Serving, Wide Expert Parallelism, Large Mixture of Experts, SGLang, vLLM, TRTLLM
InferenceMAX: Open Source Inference Benchmarking
NVIDIA GB200 NVL72, AMD MI355X, Throughput Token per GPU, Latency Tok/s/user, Perf per Dollar, Cost per Million Tokens, Tokens per Provisioned Megawatt, DeepSeek R1 670B, GPTOSS 120B, Llama3 70B