Tokens per dollar
Also known as tokens per $1 USD, tokens per RMB
In plain English
Tokens per dollar asks how many tokens one dollar of infrastructure spend buys, so a bigger number is a cheaper system.
Technical definition
Tokens per dollar is the count of tokens a configuration produces for one unit of modeled infrastructure cost, the reciprocal of cost per token.
Typical unit
tokens per $1 USD (tok/$)
Engineering details
The figure follows directly from throughput per chip and the modeled cost per chip hour, so it carries the same assumptions as cost per million tokens while reading in the direction most people reason about capacity. InferenceX publishes it for total, input, and output tokens, against each cost basis it models, and in Chinese yuan alongside US dollars.
Why it matters
Cost per million tokens and tokens per dollar rank systems identically, but a metric that rises with better hardware sits the same way up as throughput, so a chart mixing the two no longer inverts halfway down the axis. The absolute value depends entirely on the cost model behind it, so it travels only with its stated basis.
How to read it in InferenceX
Total tokens per $1 USD is the default y-axis on the InferenceX inference charts. Read it against the TCO row shown above the chart, and compare only within one cost basis: owning at hyperscaler rates, owning at neocloud rates, and three year rental produce different numbers for identical silicon.
Source material
See the concept in real benchmarks
AgentX - InferenceXv3: Does the CUDA Moat Hold Up in Agentic Inferencing?
$3 Million USD dataset open sourced, 1 Mil+ Context Length, Multiturn, Sub Agents 95%+ KVCache HitRate, GB300 NVL72, MI355X, B200
InferenceX v2: NVIDIA Blackwell Vs AMD vs Hopper - Formerly InferenceMAX
GB300 NVL72, MI355X, B200, H100, Disaggregated Serving, Wide Expert Parallelism, Large Mixture of Experts, SGLang, vLLM, TRTLLM
B200 NVFP4 vs H200 FP8 on GLM-5: Up to 3.65x Better Performance per Dollar with SGLang MTP
Both SKUs run SGLang EAGLE MTP; the Blackwell generation lifts perf/$ by ~1.2x at the peak and the NVIDIA GLM-5-NVFP4 checkpoint on FlashInfer TRT-LLM sparse MLA stacks another ~2.4–3.0x on 8K/1K