Llama 3.3 70B — H100 vs MI300X
Head-to-head AI inference benchmark comparison of H100 (NVIDIA Hopper) and MI300X (AMD CDNA 3) on Llama 3.3 70B. Latency, throughput, and cost across LLM workloads. Use the chart controls below to switch sequences, precisions, and metrics — same interactions as the main inference chart.
H100 posts 1465 tok/s/GPU for $0.25 per million tokens at 53 tok/s/user on Llama 3.3 70B; MI300X posts 1185 tok/s/GPU for $0.26. H100 is 3% cheaper per token; H100 delivers 24% more tok/s/GPU.
Throughput at 72 tok/s/user on Llama 3.3 70B: H100 hits 773 tok/s/GPU, MI300X hits 689. Per-million costs land at $0.45 and $0.46 respectively. H100 is 2% cheaper per token; H100 delivers 12% more tok/s/GPU.
H100 / MI300X on Llama 3.3 70B at 91 tok/s/user: 304 / 398 tok/s/GPU, $1.19 / $0.77 per million tokens. MI300X is 55% cheaper per token; MI300X delivers 31% more tok/s/GPU. (Numbers reflect the default 1k/1k · fp8 selection for this URL — table and chart below update if you change sequence, precision, or model in the controls.)
| Metric | Interactivity (tok/s/user) | Interactivity (tok/s/user) | Interactivity (tok/s/user) |
|---|---|---|---|
| Throughput (tok/s/gpu) | H100:1465.2MI300X:1185.2 | H100:772.9MI300X:688.6 | H100:303.7MI300X:397.9 |
| Cost ($/M tok) | H100:$0.249MI300X:$0.257 | H100:$0.454MI300X:$0.462 | H100:$1.194MI300X:$0.771 |
| tok/s/MW | H100:846932MI300X:662142 | H100:446737MI300X:384699 | H100:175535MI300X:222267 |
| Concurrency | H100:~64MI300X:~50 | H100:~50MI300X:~32 | H100:~14MI300X:~18 |
Inference Performance
Inference performance metrics across different models, hardware configurations, and serving parameters.