B200: FP4 vs FP8 Precision Comparison

FP4 posts 9789 tok/s/GPU for $0.06 per million tokens at 73 tok/s/user on DeepSeek R1 (B200); FP8 posts 2754 tok/s/GPU for $0.20. FP4 is 267% cheaper per token; FP4 delivers 255% more tok/s/GPU. Quantization-level accuracy differences are tracked on the evaluation tab.

Throughput at 136 tok/s/user on DeepSeek R1 (B200): FP4 hits 2197 tok/s/GPU, FP8 hits 1006. Per-million costs land at $0.25 and $0.54 respectively. FP4 is 119% cheaper per token; FP4 delivers 118% more tok/s/GPU. The cost-throughput tradeoff from lower precision is only part of the picture — see the evaluation page for accuracy data.

Toward the upper edge of the 11–260 tok/s/user interactivity band, at 198 tok/s/user on DeepSeek R1 (B200): FP4 runs 1412 tok/s/GPU at $0.38/M tokens, FP8 runs 373 at $1.52/M. FP4 is 297% cheaper per token; FP4 delivers 279% more tok/s/GPU. Precision changes affect both inference speed and model quality — consult the evaluation tab for accuracy benchmarks. (Numbers reflect the default 8k/1k selection for this URL — table and chart below update if you change sequence or model in the controls. Each side uses the best available serving configuration for that precision, which may include speculative decoding such as MTP where recipes exist — the same convention as the other comparison pages.)