End-to-end latency
Also known as E2EL, request completion time, P90 E2E latency
In plain English
End-to-end latency is the time from sending a model request until its complete answer has arrived, including the initial wait.
Technical definition
Request end-to-end latency measures elapsed time from request submission to receipt of the final response token.
Typical unit
seconds per completed request
Engineering details
It includes time to first token and the subsequent streaming duration. Longer answers take longer even at the same token rate, so comparisons need compatible output-length distributions. P90 E2E latency is the 90th percentile of request completion times, not a sum of separately calculated P90 stage latencies.
Why it matters
Agents often wait for a complete response before executing a tool or starting a dependent turn. Fast streaming alone does not bound that wait. Request latency also differs from the duration of a whole agent task, which can include many model calls and tool executions.
How to read it in InferenceX
The Rubin article plots P90 E2E latency separately from P90 interactivity. Read both to distinguish improved streaming cadence from reduced queueing, prefill, or overall response time.