gpt-oss 120B — H100 vs H200
Head-to-head AI inference benchmark comparison of H100 (NVIDIA Hopper) and H200 (NVIDIA Hopper) on gpt-oss 120B. Latency, throughput, and cost across LLM workloads. Use the chart controls below to switch sequences, precisions, and metrics — same interactions as the main inference chart.
Throughput at 117 tok/s/user on gpt-oss 120B: H100 hits 2622 tok/s/GPU, H200 hits 2812. Per-million costs land at $0.14 and $0.14 respectively. Cost per token is essentially tied; H200 delivers 7% more tok/s/GPU.
H100 / H200 on gpt-oss 120B at 166 tok/s/user: 1379 / 1567 tok/s/GPU, $0.26 / $0.25 per million tokens. H200 is 5% cheaper per token; H200 delivers 14% more tok/s/GPU.
Toward the upper edge of the 67–266 tok/s/user interactivity band, at 216 tok/s/user on gpt-oss 120B: H100 runs 740 tok/s/GPU at $0.49/M tokens, H200 runs 810 at $0.48/M. Cost per token is essentially tied; H200 delivers 9% more tok/s/GPU. (Numbers reflect the default 1k/1k · fp4 selection for this URL — table and chart below update if you change sequence, precision, or model in the controls.)
| Metric | Interactivity (tok/s/user) | Interactivity (tok/s/user) | Interactivity (tok/s/user) |
|---|---|---|---|
| Throughput (tok/s/gpu) | H100:2621.5H200:2812.2 | H100:1379.3H200:1567.1 | H100:739.7H200:809.8 |
| Cost ($/M tok) | H100:$0.139H200:$0.140 | H100:$0.262H200:$0.249 | H100:$0.487H200:$0.485 |
| tok/s/MW | H100:1515329H200:1625573 | H100:797303H200:905858 | H100:427578H200:468113 |
| Concurrency | H100:~64H200:~64 | H100:~17H200:~43 | H100:~8H200:~8 |
Inference Performance
Inference performance metrics across different models, hardware configurations, and serving parameters.