Qwen 3.5 397B-A17B — GB200 NVL72 vs MI300X
Head-to-head AI inference benchmark comparison of GB200 NVL72 (NVIDIA Blackwell) and MI300X (AMD CDNA 3) on Qwen 3.5 397B-A17B. Latency, throughput, and cost across LLM workloads. Use the chart controls below to switch sequences, precisions, and metrics — same interactions as the main inference chart.
Near the low end of the 36–66 tok/s/user interactivity band, at 43 tok/s/user on Qwen 3.5 397B-A17B: GB200 NVL72 runs 12698 tok/s/chip at $0.04/M tokens, MI300X runs 765 at $0.34/M. GB200 NVL72 is 745% cheaper per token; GB200 NVL72 delivers 1559% more tok/s/chip.
Setting 51 tok/s/user as the target on Qwen 3.5 397B-A17B, GB200 NVL72 produces 12313 tok/s/chip ($0.04 per million tokens) and MI300X produces 560 ($0.47). GB200 NVL72 is 1017% cheaper per token; GB200 NVL72 delivers 2101% more tok/s/chip.
At 59 tok/s/user interactivity on Qwen 3.5 397B-A17B, GB200 NVL72 delivers 11730 tok/s/chip at $0.04 per million tokens; MI300X delivers 394 tok/s/chip at $0.68. GB200 NVL72 is 1449% cheaper per token; GB200 NVL72 delivers 2879% more tok/s/chip at this point. (Numbers reflect the default 8k/1k · fp8 selection for this URL — table and chart below update if you change sequence, precision, or model in the controls.)
| Metric | Interactivity (tok/s/user) | Interactivity (tok/s/user) | Interactivity (tok/s/user) |
|---|---|---|---|
| Throughput (tok/s/chip) | GB200 NVL72:12697.9MI300X:765.3 | GB200 NVL72:12313.4MI300X:559.5 | GB200 NVL72:11730.0MI300X:393.8 |
| Cost ($/M tok) | GB200 NVL72:$0.041MI300X:$0.344 | GB200 NVL72:$0.042MI300X:$0.469 | GB200 NVL72:$0.044MI300X:$0.682 |
| tok/s/MW | GB200 NVL72:6790311MI300X:550571 | GB200 NVL72:6584710MI300X:402536 | GB200 NVL72:6272718MI300X:283284 |
| Concurrency | GB200 NVL72:~2304MI300X:~17 | GB200 NVL72:~1975MI300X:~11 | GB200 NVL72:~1447MI300X:~6 |
Inference Performance
Inference performance metrics across different models, hardware configurations, and serving parameters.