InfiniBand
Also known as IB, InfiniBand networking, NDR InfiniBand
In plain English
InfiniBand is a high speed, low latency network fabric that connects servers in AI clusters, carrying traffic between nodes that NVLink cannot reach.
Technical definition
InfiniBand is a switched network fabric with native RDMA support, used as the scale out interconnect between nodes in most large NVIDIA based AI clusters.
Engineering details
Inside a node or rack, chips talk over scale up links such as NVLink. Beyond that boundary, traffic crosses the scale out network, where InfiniBand competes with RDMA capable Ethernet. InfiniBand offers microsecond scale latency, hundreds of gigabits per second per link in current generations, and in network reduction features such as SHARP. Multi node inference, disaggregated prefill and decode, and wide expert parallelism all place their cross node collectives and KV transfers on this fabric.
Why it matters
Once a model spans nodes, the network joins the compute as a first order performance component. Fabric choice shapes cluster cost and vendor lock in, and the InfiniBand versus Ethernet contest is one of the central competitive battles in AI infrastructure.
How to read it in InferenceX
InferenceX multi node recipes, including disaggregated and rack scale results, run over the scale out fabric of the host cluster, and recipe metadata records the interconnect because it materially affects reproducibility of cross node numbers.
Source material
See the concept in real benchmarks
GB200 NVL72 vs B200 on DeepSeek R1 670B: Up to 4.4x Throughput per GPU at 125 tok/s/user
DeepSeek R1 FP4 1k/1k. NVL72's 72-GPU NVLink scale-up fabric lets decode run wide EP up to EP=32, where B200's 8-GPU NVLink island caps out at EP=8 over RoCEv2
Vera Rubin NVL72 vs GB200 NVL72? Inference TCO & Architecture Analysis
Rubin LUT Based Tensor Core, Feynman, Rack Scale, Perf Per MegaWatt, Perf Per Dollar, Software Improvements, Public Rubin Software, PyTorch, vLLM, OpenAI Triton