NVFP4
Also known as NVIDIA FP4
In plain English
NVFP4 is NVIDIA’s Blackwell-optimized version of 4-bit model math, designed to move less data and use the GPU’s fastest low-precision hardware.
Technical definition
NVFP4 is NVIDIA’s block-scaled four-bit floating-point quantization format for Blackwell-generation tensor-core inference.
Engineering details
Weights and activations are represented with compact FP4 values plus scaling information for small blocks. The exact checkpoint, scaling recipe, and kernel path determine both model quality and achieved throughput.
Why it matters
NVFP4 can reduce weight bandwidth and activate Blackwell FP4 compute paths, which is especially valuable for large MoE decode. The gain appears only when the serving engine supports the model’s attention, routing, and expert kernels end to end.
How to read it in InferenceX
InferenceX articles compare NVFP4 with FP8 or INT4 at matched interactivity. Model workload and cost assumptions stay explicit because a precision label alone cannot establish a fair benchmark.
Source material
See the concept in real benchmarks
B200 NVFP4 vs H200 FP8 on GLM-5: Up to 3.65x Better Performance per Dollar with SGLang MTP
Both SKUs run SGLang EAGLE MTP; the Blackwell generation lifts perf/$ by ~1.2x at the peak and the NVIDIA GLM-5-NVFP4 checkpoint on FlashInfer TRT-LLM sparse MLA stacks another ~2.4–3.0x on 8K/1K
B200 NVFP4 vs H100 FP8 on MiniMax-M2.5: Up to 8.2x Better Performance per Dollar with vLLM
vLLM PR #36307 unlocks the trtllm-gen FP8 MoE kernel for MiniMax on B200; combined with NVFP4, perf/$ scales from 4.0x at 22 tok/s/user to 8.2x at 110 on 8K/1K
B200 NVFP4 vs H200 INT4 on Kimi K2.5/K2.6: Up to 2.95x Better Performance per Dollar
On vLLM 8K/1K the NVFP4 path on B200 is 2.71x–2.95x cheaper per million tokens than H200 INT4 across the entire 30–90 tok/s/user serving band, and 2.45x–2.74x cheaper than B200 INT4 on the same silicon. Both factors decompose cleanly into B200's HBM bandwidth, HBM capacity, and NVFP4 tensor cores
SGLang 0.5.6 on B200 DeepSeek R1 FP4: Up to 1.8x at Low Concurrency
Piecewise CUDA graphs for DeepSeek V3, a unified event loop, and JIT kernels push 8k/1k throughput from 508 to 907 tok/s/GPU on the same 16 GPU B200 pool