E2E Normalized Interactivity
Also known as normalized interactivity, OSL/E2EL
In plain English
This metric asks how fast a whole answer arrives, counting the wait before the first word as well as the streaming speed after it.
Technical definition
E2E Normalized Interactivity is the effective per-user token rate across a complete request: output tokens divided by end to end latency.
Typical unit
tokens/second/user (tok/s/user)
Engineering details
Substituting end to end latency for time to first token plus output length times time per output token gives approximately 1 divided by the sum of inter-token latency and TTFT divided by output tokens. The result is ordinary interactivity plus a penalty proportional to first-token wait. Normalized here means normalized by output length, not scaled to a 0 to 1 score or measured against another system.
Why it matters
Interactivity alone rewards a recipe that streams quickly after making the user wait, and TTFT alone rewards one that starts fast and then crawls. Folding both into one number exposes operating points that look strong on a single axis. Short responses feel the TTFT penalty most, because there are fewer tokens to amortize the wait across.
How to read it in InferenceX
InferenceX exposes this as an experimental x-axis mode for agentic runs, which is why it needs persisted per-request traces and is unavailable for unofficial-run overlays. It is deliberately imperfect: it penalizes high TTFT heavily and does not capture every nuance of prefill and decode disaggregation, so AgentX submissions still optimize interactivity and TTFT separately.
Source material
See the concept in real benchmarks
AgentX - InferenceXv3: Does the CUDA Moat Hold Up in Agentic Inferencing?
$3 Million USD dataset open sourced, 1 Mil+ Context Length, Multiturn, Sub Agents 95%+ KVCache HitRate, GB300 NVL72, MI355X, B200
Agentic Benchmark for LLM Inference: Metrics and Methodology
How an agent benchmark replays long-context, multi-turn workloads to measure latency, throughput, cache behavior, and serving cost