Benchmark metrics
Warm-cache benchmark
In plain English
A test run with data already resident in some cache levels, which may reduce traffic to slower memory.
Technical definition
A warm-cache benchmark measures execution after prior access has populated relevant caches, with the useful residency determined by working-set size and cache behavior.
Engineering details
Repeating a small operator can reuse inputs that a production workload would fetch from a lower memory level. Warmup can also stabilize compilation and runtime state, but those effects are distinct from data-cache residency. A reproducible report identifies what was repeated and which working set fits.
Why it matters
Warm-cache tests can isolate compute or scheduling behavior while understating cold-memory costs. Comparing them with a streaming-memory test without disclosing the difference can misidentify the bottleneck.
How to read it in InferenceX
The Huawei port labels its TileLang operator comparison as warm-cache testing. That condition should accompany quoted speedups; it does not establish end-to-end serving throughput or peak HBM bandwidth.