ParallelismDCP

Decode context parallelism

Also known as DCP, KV sequence sharding

In plain English

Decode context parallelism spreads a sequence’s KV cache across ranks, so each rank reads part of the history and their partial attention results are combined.

Technical definition

Decode context parallelism shards attention KV state along the sequence dimension and merges partial attention outputs with their normalization statistics across the participating ranks.

Engineering details

This can reduce KV duplication when tensor parallelism has more ranks than independently shardable KV heads, especially with MLA’s shared latent cache. Each rank attends to its local token slice, and a numerically correct merge combines the results. The name describes its usual motivation, not a decode-only restriction: an implementation can use the same sharded cache during prefill.

Why it matters

Sharding increases effective KV capacity and distributes cache reads, but adds communication and merge work. The best degree depends on context length, attention structure, topology, and the engine’s supported combinations with tensor and prefill context parallelism.

How to read it in InferenceX

The TPU AgentX article explains DCP for GLM and Kimi K3. K3 uses the sharded layout during prefill too, avoiding the assumption that a DCP label means the prefill cache is replicated. Compare actual rank groups and cache layouts, not just the acronym.

See the concept in real benchmarks

All articles

The Iron(wood) Throne: Google's TPU Stakes Its Claim on Inference

TPUv7 AgentX results across Muse Spark Open, GLM, Qwen3.5, and Kimi K3, with TorchTPU, megakernels, and llm-d disaggregation.

AgentX - InferenceXv3: Does the CUDA Moat Hold Up in Agentic Inferencing?

$3 Million USD dataset open sourced, 1 Mil+ Context Length, Multiturn, Sub Agents 95%+ KVCache HitRate, GB300 NVL72, MI355X, B200