Trace replay
Also known as workload replay, session replay
In plain English
Trace replay recreates the timing and shape of recorded sessions so a benchmark sends requests like the original workload.
Technical definition
Trace replay is a benchmarking method that converts recorded request relationships, lengths, and timing into a repeatable workload for a system under test.
Engineering details
A replay can preserve a directed graph of main-agent turns, parallel subagent branches, and auxiliary requests. Deterministic synthetic tokens replace private content while retaining token counts and prefix relationships. Recorded gaps between turns reproduce the periods when an agent was using tools or waiting on dependencies.
Why it matters
The method captures traffic features that a list of independent prompts cannot express. It also makes repeated hardware and software comparisons possible from the same session shapes. AgentX removes source-conversation content before publishing replay data.
How to read it in InferenceX
AgentX replays trace-derived sessions through AIPerf. A fixed seed selects sessions, starting points, and synthetic content. Reported results cover the profiling window after cache warmup, which keeps run-to-run comparisons focused on steady-state serving behavior.
Source material
See the concept in real benchmarks
Ultra-High Interactivity on NVIDIA GPUs? TileRT on InferenceX
Can TileRT software on NVIDIA GPUs compete with Cerebras, Groq LPU, and SambaNova? Batch size 1, disaggregated engine, high-throughput prefill engine, high-interactivity decode engine
Vera Rubin NVL72 vs GB200 NVL72? Inference TCO & Architecture Analysis
Rubin LUT Based Tensor Core, Feynman, Rack Scale, Perf Per MegaWatt, Perf Per Dollar, Software Improvements, Public Rubin Software, PyTorch, vLLM, OpenAI Triton