Pipeline parallelism
Also known as layer parallelism, PP
In plain English
Pipeline parallelism gives each chip a different slice of the layers, passing activations along the chain instead of splitting each layer.
Technical definition
Pipeline parallelism partitions a model by layer across accelerators, so each stage runs its own layers and forwards activations to the next.
Engineering details
Communication is a point to point handoff of activations at stage boundaries, which is far cheaper than the per-layer collectives tensor parallelism requires. The cost is idle time: with one request in flight, every stage but the active one waits, and only a stream of concurrent work keeps the pipeline full.
Why it matters
For the largest models this is a capacity technique before it is a speed technique. Some frontier models do not fit in one node at all, and pipeline parallelism is what makes them servable, sometimes as the only option when a competing optimization refuses to compose with anything else.
How to read it in InferenceX
InferenceX reports the pipeline degree in point tooltips and parallelism labels alongside TP, EP, and DP, and only when it exceeds one. Composability matters as much as the degree: a stage split that blocks speculative decoding can cost more than the memory it saved.
Source material
See the concept in real benchmarks
AgentX - InferenceXv3: Does the CUDA Moat Hold Up in Agentic Inferencing?
$3 Million USD dataset open sourced, 1 Mil+ Context Length, Multiturn, Sub Agents 95%+ KVCache HitRate, GB300 NVL72, MI355X, B200
Kimi K3: The Manos, The Mythos, The Legendos
Kimi K3's architecture: compressed memory, attention across depth, latent expert routing, and serving performance
InferenceX v2: NVIDIA Blackwell Vs AMD vs Hopper - Formerly InferenceMAX
GB300 NVL72, MI355X, B200, H100, Disaggregated Serving, Wide Expert Parallelism, Large Mixture of Experts, SGLang, vLLM, TRTLLM