RadixAttention
Also known as radix tree cache, radix prefix cache
In plain English
RadixAttention keeps completed KV cache in a radix tree keyed by token content, so any new request can reuse the longest matching prefix.
Technical definition
RadixAttention is a prefix caching design from SGLang that retains KV cache entries in a radix tree after requests finish, enabling automatic reuse across requests that share token prefixes.
Engineering details
A radix tree indexes cached segments by their token content, so lookup finds the longest previously computed prefix of an incoming request in one walk. Reuse is automatic and cross request: multi turn chats, system prompts shared by many users, and agent branches that fork from a common history all hit the same cached nodes. An eviction policy such as least recently used bounds the memory the tree holds.
Why it matters
Prefix reuse converts redundant prefill compute into cache hits, and in agent traffic where each turn resends a growing history the savings can dominate end to end cost. Making reuse structural rather than opt in is a large part of why SGLang performs well on such workloads.
How to read it in InferenceX
InferenceX AgentX traces preserve real shared prefix structure between turns and subagent branches, so engines with strong radix style reuse show materially better time to first token and throughput on the agentic scenario than the fixed sequence scenarios would predict.
Source material
See the concept in real benchmarks
Agentic Benchmark for LLM Inference: Metrics and Methodology
How an agent benchmark replays long-context, multi-turn workloads to measure latency, throughput, cache behavior, and serving cost
Qwen3.5 397B on AgentX: A 20x SGLang Gap at 90 tok/s/user
GatedDeltaNet, a 262k native context, and no AMD competition at all on the same engine