GLM 5/5.1 — GB300 NVL72 vs MI355X
Head-to-head AI inference benchmark comparison of GB300 NVL72 (NVIDIA Blackwell) and MI355X (AMD CDNA 4) on GLM 5/5.1. Latency, throughput, and cost across LLM workloads. Use the chart controls below to switch sequences, precisions, and metrics — same interactions as the main inference chart.
Setting 31 tok/s/user as the target on GLM 5/5.1, GB300 NVL72 produces 3824 tok/s/GPU ($0.19 per million tokens) and MI355X produces 947 ($0.44). GB300 NVL72 is 127% cheaper per token; GB300 NVL72 delivers 304% more tok/s/GPU.
At 43 tok/s/user interactivity on GLM 5/5.1, GB300 NVL72 delivers 276 tok/s/GPU at $2.67 per million tokens; MI355X delivers 701 tok/s/GPU at $0.58. MI355X is 357% cheaper per token; MI355X delivers 154% more tok/s/GPU at this point.
GB300 NVL72 posts 93 tok/s/GPU for $7.90 per million tokens at 55 tok/s/user on GLM 5/5.1; MI355X posts 417 tok/s/GPU for $0.99. MI355X is 702% cheaper per token; MI355X delivers 348% more tok/s/GPU. (Numbers reflect the default 1k/1k · fp8 selection for this URL — table and chart below update if you change sequence, precision, or model in the controls.)
| Metric | Interactivity (tok/s/user) | Interactivity (tok/s/user) | Interactivity (tok/s/user) |
|---|---|---|---|
| Throughput (tok/s/gpu) | GB300 NVL72:3824.1MI355X:946.6 | GB300 NVL72:275.7MI355X:700.7 | GB300 NVL72:93.2MI355X:417.5 |
| Cost ($/M tok) | GB300 NVL72:$0.193MI355X:$0.438 | GB300 NVL72:$2.674MI355X:$0.585 | GB300 NVL72:$7.904MI355X:$0.985 |
| tok/s/MW | GB300 NVL72:1820984MI355X:357192 | GB300 NVL72:131293MI355X:264415 | GB300 NVL72:44379MI355X:157540 |
| Concurrency | GB300 NVL72:~6137MI355X:~128 | GB300 NVL72:~258MI355X:~40 | GB300 NVL72:~67MI355X:~16 |
Inference Performance
Inference performance metrics across different models, hardware configurations, and serving parameters.