ParallelismPCP

Prefill context parallelism

Also known as PCP, prefill sequence splitting

In plain English

Prefill context parallelism divides a long prompt’s new-token computation across ranks so one accelerator does not have to process the entire prompt alone.

Technical definition

Prefill context parallelism distributes query-token computation for a prompt across ranks, exchanging the key and value information needed to preserve the model’s attention semantics.

Engineering details

One implementation gathers keys and values while each rank owns a query chunk; another circulates KV chunks to avoid keeping the complete history on every rank. Causal attention requires load balancing because later queries can attend to more tokens. PCP’s execution partition and DCP’s KV-storage partition are related but distinct, and supported combinations vary across engines and models.

Why it matters

PCP can reduce long-prompt processing time, but additional communication, replicated weights, or a different parallel layout may consume capacity needed for KV state. The smallest prefill latency does not necessarily produce the best end-to-end serving configuration.

How to read it in InferenceX

The Kimi K3 discussion in the TPU article explains why that recipe used a DCP-sharded cache during prefill instead of its alternative PCP configuration. This is a recipe-specific memory trade-off, not a rule that PCP and DCP can never be combined.

See the concept in real benchmarks

All articles

The Iron(wood) Throne: Google's TPU Stakes Its Claim on Inference

TPUv7 AgentX results across Muse Spark Open, GLM, Qwen3.5, and Kimi K3, with TorchTPU, megakernels, and llm-d disaggregation.

AgentX - InferenceXv3: Does the CUDA Moat Hold Up in Agentic Inferencing?

$3 Million USD dataset open sourced, 1 Mil+ Context Length, Multiturn, Sub Agents 95%+ KVCache HitRate, GB300 NVL72, MI355X, B200