Hybrid attention
Also known as mixed attention, hybrid cache model
In plain English
A hybrid model mixes attention types across its layers, so its cache is several different kinds of state rather than one uniform block.
Technical definition
A hybrid attention model interleaves layers of different attention types, producing multiple KV cache groups with distinct shapes and lifetimes.
Engineering details
A uniform model has one cache layout per token, so a single block geometry describes everything worth saving. A hybrid model does not: full-attention layers, windowed layers, and recurrent or compressor state coexist, each with its own footprint and its own rules about when it can be discarded and whether it can be rebuilt.
Why it matters
The distinction is invisible until state has to leave the accelerator. A connector that assumes one uniform layout cannot say which group a block belongs to, so the models with the longest sessions were for a time the ones that could not use offload at all. Recurrent state is the hardest case, since it accumulates everything before it and cannot be recomputed from neighboring tokens.
How to read it in InferenceX
Frontier open-weight models in the InferenceX matrix are increasingly hybrid, so engine support for hybrid cache groups is part of what a recipe is measuring. Offload, disaggregated transfer, and prefix reuse each had to be extended per group rather than inherited from the uniform case.
Source material
See the concept in real benchmarks
AgentX - InferenceXv3: Does the CUDA Moat Hold Up in Agentic Inferencing?
$3 Million USD dataset open sourced, 1 Mil+ Context Length, Multiturn, Sub Agents 95%+ KVCache HitRate, GB300 NVL72, MI355X, B200
Kimi K3: The Manos, The Mythos, The Legendos
Kimi K3's architecture: compressed memory, attention across depth, latent expert routing, and serving performance
DeepSeekV4 1.6T Day 0 to Day 43 Performance Over Time — Huawei, GB300 NVL72, MI355X, B200
Day 0 Inference Performance, InferenceX, 100x performance improvement in 26 Days, Cost per Million Tokens, Huawei 950DT Inference Trace Analysis