Llama 3.3 70B · Chip comparison

Llama 3.3 70B — H200 vs MI355X

Head-to-head AI inference benchmark comparison of H200 (NVIDIA Hopper) and MI355X (AMD CDNA 4) on Llama 3.3 70B. Latency, throughput, and cost across LLM workloads. Use the chart controls below to switch sequences, precisions, and metrics — same interactions as the main inference chart.

Throughput at 37 tok/s/user on Llama 3.3 70B: H200 hits 3,361 tok/s/chip, MI355X hits 4,326. Costs per million tokens land at $0.10 and $0.096 respectively. MI355X is 4% cheaper per token; MI355X delivers 29% more tok/s/chip.

H200 / MI355X on Llama 3.3 70B at 61 tok/s/user: 2,424 / 2,613 tok/s/chip, $0.14 / $0.16 per million tokens. H200 is 12% cheaper per token; MI355X delivers 8% more tok/s/chip.

Toward the upper edge of the 14–108 tok/s/user interactivity band, at 85 tok/s/user on Llama 3.3 70B: H200 runs 1,644 tok/s/chip at $0.21 per million tokens, MI355X runs 1,060 at $0.39. H200 is 48% cheaper per token; H200 delivers 55% more tok/s/chip. (Numbers reflect the default 8k/1k · fp8 selection for this URL — table and chart below update if you change sequence, precision, or model in the controls.)

View performance per dollar →

Interpolated from real benchmark data. Edit target interactivity values below to compare at different operating points.

Metric
Interactivity (tok/s/user)
Interactivity (tok/s/user)
Interactivity (tok/s/user)
Throughput (tok/s/chip)
H2003,361MI355X4,326
H2002,424MI355X2,613
H2001,644MI355X1,060
Cost ($/M tok)
H200$0.10MI355X$0.096
H200$0.14MI355X$0.16
H200$0.21MI355X$0.39
tok/s/MW
H2002,453,643MI355X2,069,965
H2001,769,138MI355X1,250,014
H2001,200,156MI355X507,051
Concurrency
H200~32MI355X~18
H200~20MI355X~22
H200~8MI355X~6