NCCL
Also known as NVIDIA Collective Communications Library, RCCL
In plain English
NCCL is the NVIDIA library that moves data between chips during collective operations, with RCCL as its AMD counterpart.
Technical definition
NCCL is the NVIDIA collective communications library implementing operations such as all-reduce, all-gather, and all-to-all across GPUs, over NVLink within nodes and RDMA fabrics between them.
Engineering details
Frameworks do not talk to interconnects directly; they call a collectives library that discovers the topology and picks algorithms and channel schedules for each message size. NCCL handles that for NVIDIA systems, and AMD maintains RCCL with a matching interface for ROCm platforms. Tuning is fabric specific: ring versus tree algorithms, protocol thresholds, and channel counts all shift with topology, which is why the same model can communicate very differently on two clusters.
Why it matters
Every multi chip inference and training job stands on this layer, and a collectives regression can silently tax an entire fleet. Interface compatibility between NCCL and RCCL is also load bearing for portability, since engines can target one collectives API across vendors.
How to read it in InferenceX
InferenceX multi chip recipes exercise these libraries in every tensor parallel and expert parallel layer, and CollectiveX measures the underlying collective performance directly across vendors at inference relevant message sizes, separating fabric behavior from model behavior.
Source material
See the concept in real benchmarks
GB200 NVL72 vs B200 on Kimi K2.5: 3.1x from Wide EP vLLM
Rack scale NVLink on NVL72 lets Dynamo vLLM run Kimi K2.5 wide EP up to Decode EP 16, taking peak throughput from 4,021 to 12,587 tok/s/GPU on 8k/1k NVFP4
Vera Rubin NVL72 vs GB200 NVL72? Inference TCO & Architecture Analysis
Rubin LUT Based Tensor Core, Feynman, Rack Scale, Perf Per MegaWatt, Perf Per Dollar, Software Improvements, Public Rubin Software, PyTorch, vLLM, OpenAI Triton