gpt-oss 120B — H100 vs H200
Head-to-head AI inference benchmark comparison of H100 (NVIDIA Hopper) and H200 (NVIDIA Hopper) on gpt-oss 120B. Latency, throughput, and cost across LLM workloads. Use the chart controls below to switch sequences, precisions, and metrics — same interactions as the main inference chart.
Throughput at 117 tok/s/user on gpt-oss 120B: H100 hits 2622 tok/s/GPU, H200 hits 2800. Per-million costs land at $0.14 and $0.14 respectively. Cost per token is essentially tied; H200 delivers 7% more tok/s/GPU.
H100 / H200 on gpt-oss 120B at 166 tok/s/user: 1379 / 1473 tok/s/GPU, $0.26 / $0.27 per million tokens. H100 is 3% cheaper per token; H200 delivers 7% more tok/s/GPU.
Toward the upper edge of the 67–266 tok/s/user interactivity band, at 216 tok/s/user on gpt-oss 120B: H100 runs 740 tok/s/GPU at $0.49/M tokens, H200 runs 792 at $0.49/M. H100 is 2% cheaper per token; H200 delivers 7% more tok/s/GPU. (Numbers reflect the default 1k/1k · fp4 selection for this URL — table and chart below update if you change sequence, precision, or model in the controls.)
| Metric | Interactivity (tok/s/user) | Interactivity (tok/s/user) | Interactivity (tok/s/user) |
|---|---|---|---|
| Throughput (tok/s/gpu) | H100:2621.5H200:2800.1 | H100:1379.3H200:1473.4 | H100:739.7H200:791.5 |
| Cost ($/M tok) | H100:$0.139H200:$0.140 | H100:$0.262H200:$0.268 | H100:$0.487H200:$0.495 |
| tok/s/MW | H100:1515329H200:1618583 | H100:797303H200:851677 | H100:427578H200:457519 |
| Concurrency | H100:~64H200:~64 | H100:~17H200:~23 | H100:~8H200:~15 |
Inference Performance
Inference performance metrics across different models, hardware configurations, and serving parameters.