Benchmark metrics
Bytes in flight
In plain English
The amount of data requested by concurrent outstanding transfers before those transfers have all completed.
Technical definition
Bytes in flight measures outstanding transfer volume at a stated scope, such as one SM, rather than the total bytes processed over an entire benchmark.
Engineering details
More outstanding work can hide latency and raise sustained bandwidth until a resource saturates. It also consumes buffers, barrier state, and issue capacity. Thread count, load width, stage count, and transfer shape can reach the same byte total with different execution costs.
Why it matters
Equal outstanding bytes do not guarantee equal throughput or latency. Comparisons need the same scope and units, and should separate requested traffic from data delivered to multiple multicast destinations.
How to read it in InferenceX
The Blackwell port plots transfer throughput against in-flight volume. It explicitly flags places where the source prose says bytes while the chart uses KiB, rather than silently treating them as equivalent.