NVL72
Also known as GB200 NVL72, GB300 NVL72, Vera Rubin NVL72, rack-scale system
In plain English
NVL72 is a rack where 72 accelerators share one high-speed fabric, so they behave more like a single large machine than a cluster.
Technical definition
NVL72 is a rack-scale NVIDIA system that places 72 accelerators in a single NVLink scale-up domain rather than in separate eight-chip nodes.
Engineering details
NVL72 describes the size of the NVLink domain, not one fixed chip generation or bandwidth. GB200 and GB300 systems use Blackwell-generation hardware and NVLink 5; Vera Rubin pairs Rubin GPUs with Vera CPUs and NVLink 6 Switch. Compare the specific platform rather than applying Blackwell specifications to every NVL72 rack.
Why it matters
Techniques whose cost is dominated by collectives change economics inside a large domain. Wide expert parallelism spreads experts across many chips and pays all-to-all traffic for every token, which is tolerable at scale-up bandwidth and often is not across a scale-out fabric.
How to read it in InferenceX
A rack-scale advantage is not automatic. Higher cost per chip has to be earned back, and on agentic traffic the orchestration layer can become the bottleneck before the fabric does, so NVL72 configurations sometimes trail eight-chip nodes on TCO-normalized throughput for models that do not exercise wide parallelism.
Source material
See the concept in real benchmarks
GB200 NVL72 vs B200 on DeepSeek R1 670B: Up to 4.4x Throughput per GPU at 125 tok/s/user
DeepSeek R1 FP4 1k/1k. NVL72's 72-GPU NVLink scale-up fabric lets decode run wide EP up to EP=32, where B200's 8-GPU NVLink island caps out at EP=8 over RoCEv2
GB300 NVL72 vs GB200 NVL72 Inference Performance & Perf per Dollar - on DeepSeek-V4-Pro 1.6T: Up to 2.83x Throughput
DSv4-Pro FP4 8K/1K, Dynamo+vLLM, disaggregated on both racks. GB300's 50% extra HBM (288 vs 192 GB/GPU) unlocks a wider prefill+decode recipe GB200 can't fit — lifting middle-of-curve perf/$ by 2.31x despite a 20% per-GPU TCO premium.
GB200 NVL72 vs B200 on Kimi K2.5: 3.1x from Wide EP vLLM
Rack scale NVLink on NVL72 lets Dynamo vLLM run Kimi K2.5 wide EP up to Decode EP 16, taking peak throughput from 4,021 to 12,587 tok/s/GPU on 8k/1k NVFP4
Vera Rubin NVL72 vs GB200 NVL72? Inference TCO & Architecture Analysis
Rubin LUT Based Tensor Core, Feynman, Rack Scale, Perf Per MegaWatt, Perf Per Dollar, Software Improvements, Public Rubin Software, PyTorch, vLLM, OpenAI Triton
Kimi K3 on AgentX: MI355X ATOM Beats GB300 NVL72 on Part of the Curve
AMD’s vendor engine wins a real slice of the performance per dollar frontier, while Hopper struggles to serve K3 at all
OpenAI Jalapeño: Better Than Nvidia Blackwell
OpenAI’s self-designed ASIC compared with Rubin, Jalapeño’s TCO, throughput per MW, and spicy deets
Rubin NVL72 Agentic Inference: 67x better Performance per Dollar
Jensen Sandbagging Performance Again, 2x more Annual Profit Per GigaWatt, The More you Buy, The More you Earn, AgentX, InferenceX, Extreme Co-Design