Goodput
Also known as SLO-constrained throughput, useful throughput
In plain English
Goodput counts only the work that meets your latency target, so a fast-looking system that misses deadlines gets no credit for it.
Technical definition
Goodput is the portion of throughput that satisfies a stated service level objective, such as a time to first token bound or a minimum tokens per second per user.
Engineering details
Raw throughput rewards a server for finishing requests no matter how slowly each user was served. Goodput applies a filter first: a request only counts if it met the latency or interactivity constraint the operator promised. Two systems with identical throughput can have very different goodput once a deadline is applied, because one may hold latency flat under load while the other lets queueing push every request past the bound.
Why it matters
Capacity planning that ignores goodput overbuys or oversells. An operator who quotes peak throughput but serves half of it within the SLO needs twice the fleet they modeled. Goodput is the number that connects a benchmark curve to how many real users a deployment can actually carry.
How to read it in InferenceX
InferenceX publishes full throughput versus interactivity Pareto frontiers rather than a single goodput number, which lets readers apply their own SLO. Reading the frontier at a fixed interactivity tier, as the TCO calculator does, is exactly a goodput measurement at that tier.
Source material
See the concept in real benchmarks
InferenceMAX: Open Source Inference Benchmarking
NVIDIA GB200 NVL72, AMD MI355X, Throughput Token per GPU, Latency Tok/s/user, Perf per Dollar, Cost per Million Tokens, Tokens per Provisioned Megawatt, DeepSeek R1 670B, GPTOSS 120B, Llama3 70B
InferenceX v2: NVIDIA Blackwell Vs AMD vs Hopper - Formerly InferenceMAX
GB300 NVL72, MI355X, B200, H100, Disaggregated Serving, Wide Expert Parallelism, Large Mixture of Experts, SGLang, vLLM, TRTLLM