Expert parallelism
In plain English
Expert parallelism gives different GPUs different specialist parts of a model, then sends each token to the specialists it needs.
Technical definition
Expert parallelism distributes the experts of a mixture-of-experts model across accelerators and routes tokens to the ranks holding their selected experts.
Engineering details
MoE layers activate only a subset of experts for each token. EP exploits that sparsity so every GPU need not store or compute every expert, but it introduces dispatch and combine all-to-all communication around each MoE layer.
Why it matters
Wider EP reduces the expert-weight footprint per GPU and can improve decode batching and capacity. Its benefit depends on balanced routing and an interconnect fast enough to move tokens among ranks.
How to read it in InferenceX
InferenceX reports EP width as part of distributed recipes. NVL72 systems can keep much wider groups inside the NVLink scale-up domain than conventional eight-GPU nodes.
Source material
See the concept in real benchmarks
InferenceX v2: NVIDIA Blackwell Vs AMD vs Hopper - Formerly InferenceMAX
GB300 NVL72, MI355X, B200, H100, Disaggregated Serving, Wide Expert Parallelism, Large Mixture of Experts, SGLang, vLLM, TRTLLM
GB200 NVL72 vs B200 on DeepSeek R1 670B: Up to 4.4x Throughput per GPU at 125 tok/s/user
DeepSeek R1 FP4 1k/1k. NVL72's 72-GPU NVLink scale-up fabric lets decode run wide EP up to EP=32, where B200's 8-GPU NVLink island caps out at EP=8 over RoCEv2
GB200 NVL72 vs B200 on Kimi K2.5: 3.1x from Wide EP vLLM
Rack scale NVLink on NVL72 lets Dynamo vLLM run Kimi K2.5 wide EP up to Decode EP 16, taking peak throughput from 4,021 to 12,587 tok/s/GPU on 8k/1k NVFP4