Context window
Also known as context length, max sequence length, long context
In plain English
The context window is the maximum number of tokens a model can consider at once, covering both the input and everything generated so far.
Technical definition
The context window is the maximum sequence length a model supports, bounding the combined token count of prompt, conversation history, retrieved material, and generated output.
Engineering details
Attention lets every token reference earlier tokens, and the KV cache holds state for all of them, so a longer window costs memory and compute that grow with length. Position encoding schemes and training length set the usable window, while serving stacks must budget KV capacity for it. Modern frontier models advertise windows of hundreds of thousands of tokens, but throughput and interactivity degrade as sequences approach those limits.
Why it matters
Long context is what makes coding agents, retrieval heavy pipelines, and document analysis workloads possible, and it is also what makes them expensive to serve. Architectural responses such as sliding window layers, latent attention, and linear attention exist mainly to bend the cost curve of the window.
How to read it in InferenceX
InferenceX covers the window from both directions: fixed sequence scenarios pin input and output lengths such as 8K in and 1K out, while AgentX replays sessions whose contexts grow turn by turn toward realistic agent working sets.
Source material
See the concept in real benchmarks
A Brief Overview of Agentic Workloads
Multi-turn sessions, long contexts, and near-total prefix reuse make agentic inference a systems problem, and change what a benchmark has to measure
Kimi K3: The Manos, The Mythos, The Legendos
Kimi K3's architecture: compressed memory, attention across depth, latent expert routing, and serving performance