Block scaling
Also known as microscaling, MX formats, block floating point
In plain English
Block scaling gives each small group of low precision numbers its own shared scale factor, recovering range that tiny formats lack on their own.
Technical definition
Block scaling is a quantization structure where values are stored in a very low bit format and each fixed size block of them shares one higher precision scale factor.
Engineering details
A 4 bit number can represent only a handful of distinct magnitudes, far too few to span the dynamic range of model tensors. Grouping values into blocks of around 16 or 32 elements and attaching a shared scale lets each block center the format on its own magnitude. The MX standard formats such as MXFP4 and MXFP8 use power of two scales, while NVFP4 uses FP8 scales over 16 element blocks for finer granularity.
Why it matters
Block scaling is the enabling idea behind the FP4 generation of inference: without per block scales, 4 bit floating point would be unusable for frontier models. Scale format and block size choices are now genuine differentiators between hardware vendors and quantization recipes.
How to read it in InferenceX
The NVFP4 and MXFP4 results across InferenceX Blackwell and MI355X coverage are block scaled formats, and the compare-precision family exists largely to show what these recipes give up or gain against FP8 baselines on identical hardware.
Source material
See the concept in real benchmarks
B200 NVFP4 vs H200 FP8 on GLM-5: Up to 3.65x Better Performance per Dollar with SGLang MTP
Both SKUs run SGLang EAGLE MTP; the Blackwell generation lifts perf/$ by ~1.2x at the peak and the NVIDIA GLM-5-NVFP4 checkpoint on FlashInfer TRT-LLM sparse MLA stacks another ~2.4–3.0x on 8K/1K
GB300 NVL72 vs GB200 NVL72 Inference Performance & Perf per Dollar - on DeepSeek-V4-Pro 1.6T: Up to 2.83x Throughput
DSv4-Pro FP4 8K/1K, Dynamo+vLLM, disaggregated on both racks. GB300's 50% extra HBM (288 vs 192 GB/GPU) unlocks a wider prefill+decode recipe GB200 can't fit — lifting middle-of-curve perf/$ by 2.31x despite a 20% per-GPU TCO premium.