Fastest GPU for GLM-5
B300 leads at 2,988 tokens/s per GPU, 7% ahead of B200.
Measured on a single-turn chat workload (8k input / 1k output) at a matched interactivity target of 50 tokens/s per user. Hardware without a measurement at this operating point is not ranked.
| Rank | GPU | Tokens/s per GPU | $ / 1M tokens | Precision | Engine |
|---|---|---|---|---|---|
| 1 | B300 | 2,988 | $0.21 | fp4 | SGLang |
| 2 | B200 | 2,792 | $0.17 | fp4 | SGLang |
| 3 | GB200 NVL72 | 1,553 | $0.33 | fp4 | Dynamo SGLang |
| 4 | GB300 NVL72 | 1,482 | $0.43 | fp4 | Dynamo SGLang |
| 5 | MI355X | 967 | $0.43 | fp8 | SGLang |
Optimizing for cost instead of speed? See the cheapest GPU for GLM-5
Methodology
Every number on this page is a measurement, not a spec-sheet estimate. The InferenceX fleet serves GLM-5 on real hardware with community serving engines, sweeping concurrency to trace each platform's throughput-versus-interactivity frontier.
Platforms are then read at the same operating point (50 tokens/s per user) so the comparison is iso-interactivity: a GPU cannot win by quoting throughput at an unusably slow per-user speed. Cost converts measured throughput to $ per million total tokens using $/GPU/hr rates from the SemiAnalysis AI Cloud TCO model.
The derivation is shared with the InferenceX overview leaderboard, and results re-run continuously, so this ranking updates as new engine releases and configs land.
Frequently asked questions
- What is the fastest GPU for GLM-5 inference?
- As of the latest benchmark runs, B300 leads at 2,988 tokens/s per GPU on a single-turn chat workload (8k input / 1k output), measured at a matched interactivity target of 50 tokens/s per user.
- How is "fastest" measured?
- Every platform is read at the same interactivity tier (50 tokens/s per user) on a single-turn chat workload (8k input / 1k output), then ranked by measured throughput per GPU. This is the same derivation the InferenceX overview leaderboard renders, so the ranking can never disagree with the dashboard.
- How often is this ranking updated?
- Benchmarks re-run continuously on the InferenceX cluster fleet; the newest result feeding this ranking landed on 2026-08-24. The page always renders the latest data.