GLM 5/5.1 · B300 · Precision comparison

B300: FP4 vs FP8 precision comparison

How FP4 and FP8 precision affect GLM 5/5.1 inference on B300 (NVIDIA Blackwell). Throughput, latency, and cost across LLM workloads. Use the chart controls below to switch sequences and metrics — same interactions as the main inference chart.

FP4 posts 1,775 tok/s/chip for $0.35 per million tokens at 51 tok/s/user on GLM 5/5.1 (B300); FP8 posts 671 tok/s/chip for $0.94. FP4 delivers 164% more tok/s/chip on the same chip, which makes it 62% cheaper per token. Quantization-level accuracy differences are tracked on the evaluation tab.

Throughput at 80 tok/s/user on GLM 5/5.1 (B300): FP4 hits 1,034 tok/s/chip, FP8 hits 424. Costs per million tokens land at $0.61 and $1.48 respectively. FP4 delivers 144% more tok/s/chip on the same chip, which makes it 59% cheaper per token. The cost-throughput tradeoff from lower precision is only part of the picture — see the evaluation page for accuracy data.

Toward the upper edge of the 24–136 tok/s/user interactivity band, at 108 tok/s/user on GLM 5/5.1 (B300): FP4 runs 563 tok/s/chip at $1.11 per million tokens, FP8 runs 235 at $2.67. FP4 delivers 139% more tok/s/chip on the same chip, which makes it 58% cheaper per token. Precision changes affect both inference speed and model quality — consult the evaluation tab for accuracy benchmarks. (Numbers reflect the default 1k/1k selection for this URL — table and chart below update if you change sequence or model in the controls. Each side uses the best available serving configuration for that precision, which may include speculative decoding such as MTP where recipes exist — the same convention as the other comparison pages.)

Interpolated from real benchmark data. Edit target interactivity values below to compare at different operating points.

Metric
Interactivity (tok/s/user)
Interactivity (tok/s/user)
Interactivity (tok/s/user)
Throughput (tok/s/chip)
FP41,775FP8671
FP41,034FP8424
FP4563FP8235
Cost ($/M tok)
FP4$0.35FP8$0.94
FP4$0.61FP8$1.48
FP4$1.11FP8$2.67
tok/s/MW
FP4934,379FP8353,288
FP4544,052FP8223,210
FP4296,573FP8123,852
Concurrency
FP4~73FP8~54
FP4~26FP8~22
FP4~11FP8~9
GLM 5/5.1: FP4 versus FP8 throughput and cost at matched interactivity levels on B300
FP4 versus FP8 throughput and cost per million tokens on B300 for this comparison's canonical default workload
Open full-size image ↗