Benchmark metrics

Performance per dollar

Also known as perf/$, cost efficiency

In plain English

Performance per dollar measures how much useful AI output the system produces for each dollar spent running it.

Technical definition

Performance per dollar expresses how much measured inference work a system delivers for a unit of modeled cost.

Engineering details

For a fixed workload and interactivity target, performance per dollar is the inverse of cost per token. A 2× perf/$ advantage means the system can produce about twice as many comparable tokens for the same infrastructure spend.

Why it matters

Peak chip FLOPS account for only part of serving economics. Memory, networking, software maturity, numerical precision, and achievable utilization all affect the measured output behind the ratio.

How to read it in InferenceX

InferenceX compares infrastructure perf/$ at matched interactivity and names the TCO inputs used. The Rubin article’s approximately 67x result is scoped to 170 TPS, owning costs, and the specified TRTLLM NVFP4 Dense configurations. It is not a generation-wide multiplier: the ratio changes with the target, comparison engine, and owning versus rental cost basis.

Source material

See the concept in real benchmarks

All articles →

B200 NVFP4 vs H200 FP8 on GLM-5: Up to 3.65x Better Performance per Dollar with SGLang MTP

Both SKUs run SGLang EAGLE MTP; the Blackwell generation lifts perf/$ by ~1.2x at the peak and the NVIDIA GLM-5-NVFP4 checkpoint on FlashInfer TRT-LLM sparse MLA stacks another ~2.4–3.0x on 8K/1K

B200 NVFP4 vs H100 FP8 on MiniMax-M2.5: Up to 8.2x Better Performance per Dollar with vLLM

vLLM PR #36307 unlocks the trtllm-gen FP8 MoE kernel for MiniMax on B200; combined with NVFP4, perf/$ scales from 4.0x at 22 tok/s/user to 8.2x at 110 on 8K/1K

B200 NVFP4 vs H200 INT4 on Kimi K2.5/K2.6: Up to 2.95x Better Performance per Dollar

On vLLM 8K/1K the NVFP4 path on B200 is 2.71x–2.95x cheaper per million tokens than H200 INT4 across the entire 30–90 tok/s/user serving band, and 2.45x–2.74x cheaper than B200 INT4 on the same silicon. Both factors decompose cleanly into B200's HBM bandwidth, HBM capacity, and NVFP4 tensor cores

AMD MI355X GLM-5 Inference: Up to 40% Cheaper per Million Tokens than B200 on SGLang FP8

14 weeks after GLM-5 launched, AMD landed both MTP and non-MTP SGLang FP8 recipes on MI355X — fused MLA + FP8 KV cache via TileLang flips the single-node FP8 cost curve in AMD favor across most of the performance Pareto

DeepSeek V4 Pro on AgentX: MI355X vs B200, and the August 21 Flip

AMD matched B200 vLLM on performance per dollar for end-to-end latency, then upstream vLLM work moved the line

TPU Inference Externalization Full Steam Ahead

InferenceX, Up to 50% Better Performance per Dollar, Rapid Externalization of TPU stack, Growing Customer Base, Ironwood, TPUv8i, Reducing CUDA Moat

Rubin NVL72 Agentic Inference: 67x better Performance per Dollar

Jensen Sandbagging Performance Again, 2x more Annual Profit Per GigaWatt, The More you Buy, The More you Earn, AgentX, InferenceX, Extreme Co-Design