gpt-oss 120B — H100 vs MI300X
Head-to-head AI inference benchmark comparison of H100 (NVIDIA Hopper) and MI300X (AMD CDNA 3) on gpt-oss 120B. Latency, throughput, and cost across LLM workloads. Use the chart controls below to switch sequences, precisions, and metrics — same interactions as the main inference chart.
Setting 112 tok/s/user as the target on gpt-oss 120B, H100 produces 2796 tok/s/GPU ($0.13 per million tokens) and MI300X produces 1599 ($0.20). H100 is 50% cheaper per token; H100 delivers 75% more tok/s/GPU.
At 158 tok/s/user interactivity on gpt-oss 120B, H100 delivers 1521 tok/s/GPU at $0.24 per million tokens; MI300X delivers 803 tok/s/GPU at $0.37. H100 is 57% cheaper per token; H100 delivers 90% more tok/s/GPU at this point.
H100 posts 878 tok/s/GPU for $0.41 per million tokens at 203 tok/s/user on gpt-oss 120B; MI300X posts 309 tok/s/GPU for $0.97. H100 is 138% cheaper per token; H100 delivers 184% more tok/s/GPU. (Numbers reflect the default 1k/1k · fp4 selection for this URL — table and chart below update if you change sequence, precision, or model in the controls.)
| Metric | Interactivity (tok/s/user) | Interactivity (tok/s/user) | Interactivity (tok/s/user) |
|---|---|---|---|
| Throughput (tok/s/gpu) | H100:2795.7MI300X:1598.8 | H100:1521.3MI300X:802.8 | H100:878.3MI300X:309.1 |
| Cost ($/M tok) | H100:$0.131MI300X:$0.195 | H100:$0.238MI300X:$0.373 | H100:$0.409MI300X:$0.975 |
| tok/s/MW | H100:1616027MI300X:893159 | H100:879363MI300X:448476 | H100:507706MI300X:172687 |
| Concurrency | H100:~64MI300X:~15 | H100:~28MI300X:~5 | H100:~10MI300X:~7 |
Inference Performance
Inference performance metrics across different models, hardware configurations, and serving parameters.