Llama 3.3 70B — H200 vs MI325X
Head-to-head AI inference benchmark comparison of H200 (NVIDIA Hopper) and MI325X (AMD CDNA 3) on Llama 3.3 70B. Latency, throughput, and cost across LLM workloads. Use the chart controls below to switch sequences, precisions, and metrics — same interactions as the main inference chart.
Setting 51 tok/s/user as the target on Llama 3.3 70B, H200 produces 3169 tok/s/GPU ($0.12 per million tokens) and MI325X produces 1541 ($0.23). H200 is 87% cheaper per token; H200 delivers 106% more tok/s/GPU.
At 75 tok/s/user interactivity on Llama 3.3 70B, H200 delivers 2062 tok/s/GPU at $0.19 per million tokens; MI325X delivers 841 tok/s/GPU at $0.42. H200 is 119% cheaper per token; H200 delivers 145% more tok/s/GPU at this point.
H200 posts 1180 tok/s/GPU for $0.33 per million tokens at 100 tok/s/user on Llama 3.3 70B; MI325X posts 437 tok/s/GPU for $0.80. H200 is 140% cheaper per token; H200 delivers 170% more tok/s/GPU. (Numbers reflect the default 1k/1k · fp8 selection for this URL — table and chart below update if you change sequence, precision, or model in the controls.)
| Metric | Interactivity (tok/s/user) | Interactivity (tok/s/user) | Interactivity (tok/s/user) |
|---|---|---|---|
| Throughput (tok/s/gpu) | H200:3169.1MI325X:1540.8 | H200:2062.3MI325X:840.8 | H200:1179.9MI325X:436.9 |
| Cost ($/M tok) | H200:$0.124MI325X:$0.231 | H200:$0.190MI325X:$0.416 | H200:$0.335MI325X:$0.802 |
| tok/s/MW | H200:1831840MI325X:706774 | H200:1192078MI325X:385669 | H200:682027MI325X:200418 |
| Concurrency | H200:~64MI325X:~64 | H200:~63MI325X:~53 | H200:~24MI325X:~18 |
Inference Performance
Inference performance metrics across different models, hardware configurations, and serving parameters.