Tutorial
Every GPU. Every second. Every watt.
PowerX measures the power behind AI inference, in the open.
Updated
Loading the measured point…
On an HGX chassis the exporter reports GPU board power. Two x86 CPUs, 32 DIMMs, eight NICs, four PCIe switches, ten NVMe drives, fans and PSU loss are modeled around it, and PUE 1.3 covers air cooling.
- Measured
- Modeled
- Rated or assumed
PowerX measures what each accelerator draws while it serves a benchmark and models the rest of the system around it. Solid marks are measured telemetry, outlined marks are modeled by the open-source power model, dashed marks are rated or assumed values. The grammar holds on every chart.
From measurement to modeled power
Measured GPU W. Average, P75 and P90 watts per GPU are time-weighted over the window across every GPU in the deployment; prefill and decode roles are averaged over the same window.
- Measured
- Modeled
- Rated or assumed
- Sampled.
dcgm-exporteron NVIDIA nodes anddevice-metrics-exporteron AMD nodes report every GPU once a second; the head node scrapes every node on one clock. - Windowed. Each concurrency point has one serving window, from the end of warmup to the last request. Only samples inside it count.
- Integrated. Energy per GPU is the trapezoid integral of its samples over the window; energy per token divides the deployment's GPU energy by the tokens the client counted.
- Modeled. The
power_modelpackage takes the measured watts and an operating state (fixed sequence, agentic, agentic with KV offload), adds CPUs, memory, NICs, switches, storage, fans and PSU loss to reach IT power, then applies PUE (1.3 for an air-cooled chassis, 1.1 for a liquid-cooled NVL72 rack) for facility power.
Explore PowerX
The modeled curve sits above the measured one with the same shape, because the parts the model adds barely move with load.
- Measured
- Modeled
Loading validated points…
Lower is better, and both lines are Pareto frontiers against interactivity like the rest of the dashboard.
Move the GPU from idle to TDP to see which modeled parts follow it.
B200 · IT 881 W per GPU
GPU
600 Wmeasured input
Fans
29 Wfollows the GPU
Network share
51 Wflat
NVSwitch
50 Wflat
PSU loss
35 Wfollows the GPU
CPU
38 Wflat
PCIe & misc
35 Wflat
NICs
23 Wflat
Memory
15 Wflat
Storage
6 Wflat
Fans and PSU loss track the GPU; CPUs, memory, NICs and switches hold their level, so a lightly loaded GPU carries the largest overhead in relative terms.
Methodology
Average, P75 and P90 power are time-weighted over the window across every GPU in the deployment; prefill and decode roles are averaged over the same window. Planners read P90, because dynamic power steering reacts to peaks; chip comparisons read energy per token.
A point is Validated only when the offline validator, re-run from the raw samples, passes all five checks. Anything else is Historical: throughput stays, energy and modeled power are dropped, and the audit record names the failing check.
01Every expected GPU reported for the whole window
The validator lists the devices the topology expects and the devices that appear in samples.csv; one missing GPU fails the point.
Recorded as
expected_device_missing02No gap between samples longer than 3 s
A stalled exporter leaves holes the trapezoid integral would paper over, so the largest per-GPU gap is checked against the limit.
Recorded as
sample_gap_exceeded03Telemetry window matches the client window within 0.5 s or 1 %
The head node writes the window from its own clock; the benchmark client reports its own start and end. The two must agree within max(0.5 s, 1 % of the window).
Recorded as
measurement_window_clock_mismatch04Producer commit matches the pinned version, exporter image recorded
The code that wrote the samples is identified by its git commit and must equal the pin the validator was built with; the exporter image digest is stored beside it.
Recorded as
producer_commit_mismatch05Token counts the energy is divided by are positive
Joules per token needs a denominator. A window with zero output tokens keeps its watts but cannot carry an energy metric.
Recorded as
invalid_output_token_count
All five pass: Validated. Any failure: Historical, throughput kept, energy and modeled power dropped.
Power collection is part of every official sweep, so a submitted recipe is measured automatically. To run it yourself, add power: true to a multi-node matrix entry; single-node and AgentX jobs use the cluster's native exporter. require-power (or REQUIRE_POWER=1) fails the job on an invalid window instead of recording power_valid: 0.
More options
kimik3-fp4-h200-dynamo-sglang:
multinode: true
scenarios:
fixed-seq-len:
- isl: 8192
osl: 1024
search-space:
- num-nodes: 4
srt-recipe: agg-tp16dp2ep32-latency.yaml
power: true
conc-list: [1, 2, 4, 8]The aggregate JSON carries power_valid, avg_power_w, p90_power_w and the joules_per_* metrics; the power_audit_<result> artifact holds samples.csv, the window files and power_validation_<result>.json.
Apply the results
One validated point, read the way a capacity plan reads it: measured watts, the IT watts the model adds, GPUs per megawatt, and the rated marks it sits under.
Loading the measured point…
Open PowerX, pull the same numbers from the API, or run inferencex powerx export from the skills package.
FAQ
Why is measured power so far below TDP?
Decode-heavy inference is bound by memory and latency targets, not by compute, so the GPU rarely reaches its power limit. Prefill, video generation and training sit close to TDP.
Do DCGM and nvidia-smi disagree?
No. Both read the same NVML sensor, and the firmware already averages over roughly a second, which is why sampling faster than 1 Hz adds little. DCGM adds a daemon and an energy counter.
Why P90 rather than average?
Capacity is provisioned for peaks. P90 is the power exceeded for 10 % of the window and is what a buffer should be added to.
Why offer IT without PUE?
Data-center capacity teams size on critical IT kW per GPU. PUE depends on the site, so Facility applies it separately.
What happens when the power model changes?
Nothing is stored. Every modeled number is recomputed from the stored GPU measurements with the pinned model revision, so an update changes every point at once and the revision is shown in the tooltip.
Why is one point Historical when the rest of the run is Validated?
Validity is per window. One lost sample or a clock mismatch invalidates that window only; the point keeps its throughput and loses its energy.
Does a higher Facility number mean the site is less efficient?
No. PUE is a fixed model input per cooling type, not a measurement of the benchmark site.