Llama 3.3 70B · Chip comparison

Llama 3.3 70B — H100 vs MI325X

Head-to-head AI inference benchmark comparison of H100 (NVIDIA Hopper) and MI325X (AMD CDNA 3) on Llama 3.3 70B. Latency, throughput, and cost across LLM workloads. Use the chart controls below to switch sequences, precisions, and metrics — same interactions as the main inference chart.

At 38 tok/s/user interactivity on Llama 3.3 70B, H100 delivers 1,848 tok/s/chip at $0.18 per million tokens; MI325X delivers 1,696 tok/s/chip at $0.18. H100 is 2% cheaper per token; H100 delivers 9% more tok/s/chip at this point.

H100 posts 1,171 tok/s/chip for $0.28 per million tokens at 58 tok/s/user on Llama 3.3 70B; MI325X posts 1,354 tok/s/chip for $0.23. MI325X is 19% cheaper per token; MI325X delivers 16% more tok/s/chip.

Throughput at 78 tok/s/user on Llama 3.3 70B: H100 hits 877 tok/s/chip, MI325X hits 887. Costs per million tokens land at $0.37 and $0.34 respectively. MI325X is 7% cheaper per token; MI325X delivers 1% more tok/s/chip. (Numbers reflect the default 8k/1k · fp8 selection for this URL — table and chart below update if you change sequence, precision, or model in the controls.)

View performance per dollar →

Interpolated from real benchmark data. Edit target interactivity values below to compare at different operating points.

Metric
Interactivity (tok/s/user)
Interactivity (tok/s/user)
Interactivity (tok/s/user)
Throughput (tok/s/chip)
H1001,848MI325X1,696
H1001,171MI325X1,354
H100877MI325X887
Cost ($/M tok)
H100$0.18MI325X$0.18
H100$0.28MI325X$0.23
H100$0.37MI325X$0.34
tok/s/MW
H1001,348,638MI325X1,003,526
H100855,073MI325X801,393
H100640,410MI325X524,995
Concurrency
H100~20MI325X~25
H100~16MI325X~25
H100~8MI325X~11