Llama 3.3 70B — H100 vs MI325X
Head-to-head AI inference benchmark comparison of H100 (NVIDIA Hopper) and MI325X (AMD CDNA 3) on Llama 3.3 70B. Latency, throughput, and cost across LLM workloads. Use the chart controls below to switch sequences, precisions, and metrics — same interactions as the main inference chart.
At 53 tok/s/user interactivity on Llama 3.3 70B, H100 delivers 1465 tok/s/GPU at $0.25 per million tokens; MI325X delivers 1467 tok/s/GPU at $0.24. MI325X is 3% cheaper per token; throughput per GPU is essentially tied at this point.
H100 posts 773 tok/s/GPU for $0.45 per million tokens at 72 tok/s/user on Llama 3.3 70B; MI325X posts 899 tok/s/GPU for $0.39. MI325X is 16% cheaper per token; MI325X delivers 16% more tok/s/GPU.
Throughput at 91 tok/s/user on Llama 3.3 70B: H100 hits 304 tok/s/GPU, MI325X hits 572. Per-million costs land at $1.19 and $0.62 respectively. MI325X is 93% cheaper per token; MI325X delivers 88% more tok/s/GPU. (Numbers reflect the default 1k/1k · fp8 selection for this URL — table and chart below update if you change sequence, precision, or model in the controls.)
| Metric | Interactivity (tok/s/user) | Interactivity (tok/s/user) | Interactivity (tok/s/user) |
|---|---|---|---|
| Throughput (tok/s/gpu) | H100:1465.2MI325X:1467.1 | H100:772.9MI325X:899.3 | H100:303.7MI325X:571.6 |
| Cost ($/M tok) | H100:$0.249MI325X:$0.243 | H100:$0.454MI325X:$0.391 | H100:$1.194MI325X:$0.618 |
| tok/s/MW | H100:846932MI325X:672962 | H100:446737MI325X:412536 | H100:175535MI325X:262183 |
| Concurrency | H100:~64MI325X:~64 | H100:~50MI325X:~59 | H100:~14MI325X:~26 |
Inference Performance
Inference performance metrics across different models, hardware configurations, and serving parameters.