AIPerf
Also known as replay harness, load generator
In plain English
AIPerf is the vendor-neutral client that sends the benchmark traffic, reconstructing recorded agent sessions and timing every request.
Technical definition
AIPerf is the open-source HTTP load generation and replay harness that drives AgentX runs against a serving endpoint.
Engineering details
It reconstructs each session as a directed acyclic graph in which nodes are requests and edges carry the delay before a dependent request may be sent. That structure reproduces main-agent turns, parallel subagent branches that join later, one-off auxiliary requests, and the tool-use pauses between turns, none of which a flat list of prompts can express.
Why it matters
The client is part of the measurement. A harness that cannot express dependencies would issue an agentic workload as independent requests and erase exactly the burstiness and reuse the scenario exists to test. Keeping it vendor neutral also keeps the load generator from favoring any one serving stack.
How to read it in InferenceX
A seed fixes which conversations are sampled, where each starts, and the synthetic content used to fill anonymized blocks, so a rerun of the same recipe replays the same workload. All submissions for a model run the same harness minor version so results stay comparable.
Source material
See the concept in real benchmarks
AgentX - InferenceXv3: Does the CUDA Moat Hold Up in Agentic Inferencing?
$3 Million USD dataset open sourced, 1 Mil+ Context Length, Multiturn, Sub Agents 95%+ KVCache HitRate, GB300 NVL72, MI355X, B200
Agentic Benchmark for LLM Inference: Metrics and Methodology
How an agent benchmark replays long-context, multi-turn workloads to measure latency, throughput, cache behavior, and serving cost
A Brief Overview of Agentic Workloads
Multi-turn sessions, long contexts, and near-total prefix reuse make agentic inference a systems problem, and change what a benchmark has to measure