Public data API
InferenceX API reference
Read benchmark, provenance, dataset, CollectiveX, and diagnostic data from the same sources that power InferenceX.
Machine-readable contract
OpenAPI 3.1 JSON
Inspect the canonical schema or pass it directly to your tooling.
Open OpenAPI JSON- Specification
- v1 · OpenAPI 3.1
- Authentication
- Published read endpoints do not require authentication.
- Response format
- Responses are JSON unless an endpoint explicitly documents CSV. Dates use YYYY-MM-DD and timestamps use UTC ISO 8601.
- Base URL
- https://inferencex.semianalysis.com
Quickstart
Move from contract discovery to a real response in a few steps.
- 1
Discover configurations
Start with availability to choose real model, hardware, framework, and sequence values.
Discover configurationscurl "https://inferencex.semianalysis.com/api/v1/availability" - 2
Fetch benchmark rows
Then request the latest raw benchmark rows for a supported display model.
Fetch benchmark rowscurl "https://inferencex.semianalysis.com/api/v1/benchmarks?model=DeepSeek-R1-0528"
Skill version · 1.2.0
Use the API with your agent
The inferencex-api skill helps your agent navigate the public API: benchmarks, provenance, datasets, CollectiveX, and diagnostics. Validated single-turn PowerX export is the first worked example.
@semianalysisai/inferencex-skills
Requires Node 24 or later with npm and Codex or Claude Code. Installation and API queries require internet access.
Queries use the public API without database credentials. The separate MCP server has its own setup; this skill does not require or reconfigure it.
Install in your project
Run the command for your agent from your project directory, then start an agent session in that project.
Codex
npm exec --yes --package @semianalysisai/inferencex-skills@1.2.0 -- inferencex-skills install --target codexClaude Code
npm exec --yes --package @semianalysisai/inferencex-skills@1.2.0 -- inferencex-skills install --target claudeTo upgrade, replace the version in the installation command with a new published version and rerun it with --force. Existing skills are otherwise skipped. Save local edits first: --force overwrites matching files and retains obsolete files.
To check the copied skill version, replace install with status in the command above and keep the same --target. Installer version identifies the installer; Installed version identifies the copied skill. Older installations may report unknown.
Usage examplesFirst example: measured PowerX
Use inferencex-api to export latest available measured PowerX data for DeepSeek-V4-Pro:
- Select single-turn requests with exactly 8192 input and 1024 output tokens; require strictV2.
- Use the installed CLI's powerx export command to create separate CSV and JSON evidence bundles outside the InferenceX repository. Keep each result.csv or result.json with its manifest.json and responses/, and verify each bundle before reporting.
- Keep measured per-GPU watts and whole-deployment GPU energy distinct from provisioned-power estimates.
- Preserve raw model keys, source IDs/URLs, measurement dates, and separate snapshot metadata.
- Record the request URL, retrieval time, package version, local filters, and returned/selected counts.
- Explain why rows were excluded and list missing requested metrics.
- Keep missing metrics unavailable and genuine zeros unchanged. Explain empty results without inferring that all benchmarks are absent.PowerX cookbook and direct export
Open the cookbook at the installed path below for validity, units, missing-data handling, and provenance. The bundled Node 24 CLI runs directly from your project, without an InferenceX checkout or database credentials.
- Codex
- .agents/skills/inferencex-api/references/powerx.md
- Claude Code
- .claude/skills/inferencex-api/references/powerx.md
Run the installed CLI (Codex)
mkdir -p evidence
node .agents/skills/inferencex-api/scripts/inferencex.mjs powerx export \
--model DeepSeek-V4-Pro --isl 8192 --osl 1024 \
--format csv --output-dir evidence/powerx-csv
node .agents/skills/inferencex-api/scripts/inferencex.mjs verify evidence/powerx-csvFor Claude Code, use the .claude/skills/inferencex-api path. For JSON, use --format json --output-dir evidence/powerx-json and verify that directory. Use a new output directory for each export. It contains result.csv or result.json, manifest.json, and responses/; keep the whole bundle to preserve request and coverage metadata even for an empty result. Omit --date for latest available observations, or add --date YYYY-MM-DD for an as-of cutoff.
avg_power_w is measured mean watts per GPU. Schema-v2 joules metrics without a role prefix describe whole-deployment GPU energy; prefill/decode-prefixed energy is role-local. These are existing observations, not new benchmark runs or facility-energy measurements.
Conventions
Shared request, error, and cache behavior for the supported surface.
- Client request counts
- Optional inferencex-cli/<version> or inferencex-skill/<version> User-Agent markers attribute public GET arrivals, including cache hits and retries. X-InferenceX-Traffic separates normal, ci and validation requests. Only route templates, package versions, transport, environment and traffic class are recorded; no query values or user identifiers. Packaged clients can omit attribution with INFERENCEX_TELEMETRY=0. These headers do not affect authentication, response data or caching.
- Errors
- JSON errors contain an error string. A 400 response means a parameter is missing or invalid, 404 means the requested record is absent, and 500 means the server query failed.
- Caching
- Read endpoints may be served from shared caches. JSON is brotli- or gzip-compressed when Accept-Encoding allows it and uncompressed otherwise, with Vary: Accept-Encoding. /api/v1/benchmarks is the exception: the shared cache compresses it, so its ETag and Vary headers come from that cache. Most database-backed JSON responses carry a strong ETag; send it in If-None-Match to receive an empty 304 while the data is unchanged. Browsers may reuse those responses for 60 seconds. Right after a data ingest, a cache may serve the previous response once while it refreshes. CollectiveX responses stay cached until a results upload purges them, and framework releases use a one-hour shared cache.
- Identifiers
- Benchmark result IDs and GitHub run IDs are positive integers. Bulk diagnostic endpoints accept comma-separated, deduplicated IDs.
BenchmarkRow and metrics
Interpret the primary benchmark payload and its measured fields.
- DWDP prefill topology
Optional metrics.prefill_dwdp_size records ranks per distributed weight-data parallel prefill group. prefill_tp stays 1; prefill_num_workers counts groups. Two groups of size 8 use 16 prefill chips. Sibling rows expose nullable prefill_dwdp_size. Missing or 1 means DWDP is disabled.
ShapeBenchmarkRowsExample{ "prefill_tp": 1, "prefill_num_workers": 2, "metrics": { "prefill_dwdp_size": 8 } }- BenchmarkRow
Configuration fields sit beside a metrics map. Time metrics are seconds. Throughput fields ending in _per_gpu retain their legacy names but measure tokens per second per physical chip, including TPUs. num_prefill_gpu and num_decode_gpu count physical chips independently of logical TP; aggregate engines may mirror one count in both columns, so sum them only for disaggregated rows with nonzero prefill_num_workers and decode_num_workers or unequal counts; legacy disaggregated rows without workers (2025-09-29 to 2025-12-05) mirror the deployment size in both columns too. Optional metrics.dp records data parallelism when supplied. For a P99 inter-token latency requirement, compare p99_itl in seconds (multiply by 1000 for milliseconds). p99_tpot measures per-request time per output token; the reciprocal of p99_intvty is not a substitute for p99_itl.
ShapeBenchmarkRowsExample{ "id": 421, "hardware": "h200_sxm", "framework": "vllm", "model": "dsr1", "precision": "fp8", "spec_method": "none", "disagg": false, "is_multinode": false, "prefill_tp": 8, "prefill_ep": 1, "prefill_dp_attention": false, "prefill_num_workers": 1, "decode_tp": 8, "decode_ep": 1, "decode_dp_attention": false, "decode_num_workers": 1, "num_prefill_gpu": 0, "num_decode_gpu": 8, "benchmark_type": "single_turn", "isl": 1024, "osl": 1024, "conc": 32, "offload_mode": "off", "image": "vllm/vllm-openai:v0.10.2", "recipe_fingerprint": "7d72a33d7d72a33d7d72a33d7d72a33d7d72a33d7d72a33d7d72a33d7d72a33d", "metrics": { "median_ttft": 0.42, "median_tpot": 0.018, "tput_per_gpu": 128.4, "power_valid": 1, "power_metric_schema_version": 2, "avg_power_w": 678.5, "joules_per_output_token": 5.3, "joules_per_total_token": 2.65, "avg_temp_c": 61.2 }, "date": "2026-08-08", "run_url": "https://github.com/semianalysis/inference-benchmarks/actions/runs/123456789" }Scroll to inspect the full example
- Measured power
Benchmark rows may carry measured power, energy, and GPU-telemetry metric keys (avg_power_w, avg_total_gpu_power_w, total_gpu_energy_j, p75_power_w, p75_total_gpu_power_w, p90_power_w, p90_total_gpu_power_w, joules_per_*, avg_temp_c, peak_temp_c, avg_util_pct, avg_mem_used_mb). power_valid is tri-state: 1 means the measurement window was validated; 0 means validation failed and measured values are withheld end-to-end (the producer strips them and ingest scrubs them — treat any that remain as unreliable); absent means no validation verdict is available in this response. Legacy rows can lack the field, but absence alone establishes neither the reason, the measurement age, nor invalidity. power_metric_schema_version == 2 defines every unprefixed joules_per_* field as whole-deployment energy — unversioned disaggregated joules are ambiguous because those fields previously carried role-local values. workers[] carries the per-worker power/telemetry breakdown on multinode and disaggregated runs. power_invalid_reasons lists producer reason codes. power_audit optionally carries measurement-window bounds, device and sample counts, producer identity and the retained audit reference on valid and invalid rows. Missing audit metadata does not establish the measurement age or validity. NVL72 (GB200/GB300) rows may add Grace-side and compute-module keys integrated over the same window: avg_cpu_socket_power_w, avg_total_cpu_power_w, total_cpu_energy_j, avg_total_module_power_w / total_module_energy_j when the module sensor exists on every socket, and the CPU core and SysIO rail breakdowns avg_total_cpu_rail_power_w / total_cpu_rail_energy_j and avg_total_cpu_sysio_power_w / total_cpu_sysio_energy_j when every socket reports them. The rails are inside the Grace-side total; never add them to it. Single-node AMD MI300X/MI325X/MI355X rows may add only avg_cpu_socket_power_w, avg_total_cpu_power_w and total_cpu_energy_j, read from the EPYC RAPL package energy counter (sensor_kind rapl_package), which has no DRAM domain. cpu_power_valid is their own tri-state verdict, independent of power_valid: the CPU-side keys are withheld on cpu_power_valid != 1, not on power_valid, and power_audit.cpu records the sensor kind, collector, socket coverage and reason codes. A module reading already contains the GPU-board watts. For measured-power requests, use powerValid=strictV2 to require power_valid == 1 and power_metric_schema_version == 2. It is the only supported power filter. Omit powerValid for general benchmark requests so results remain available even when they lack valid power measurements.
ShapeBenchmarkRowsExample{ "power_valid": 1, "power_metric_schema_version": 2, "avg_power_w": 678.5, "joules_per_output_token": 5.3, "cpu_power_valid": 1, "avg_cpu_socket_power_w": 250.5, "avg_total_cpu_power_w": 501 }- ID-keyed maps
Bulk diagnostic responses are JSON objects whose keys are decimal benchmark result IDs. A missing key means no value was available for that ID.
ShapeRecord<string, value>Example{ "421": true }- CollectiveX versions
CollectiveX reads require an explicit supported contract version. Supported versions: 1.
ShapeCollectiveXDataset
Endpoint reference
Expand an operation for parameters, statuses, and complete examples.
54 operations
Read-only dashboard views
Dashboard data and calculations with explicit filters.
GETBeta/api/v1/views/optionsDiscover every views-API option domain
Static discovery endpoint for the views API: models (with DB keys, category and release date), sequences, precisions, hardware (with vendor, architecture, TDP and per-provider hourly cost), frameworks, speculative-decoding methods, percentiles, x-axis and scale modes, chart metrics (bilingual labels, unit, polarity), quick-filter domains, reliability ranges, and overview/calculator/fleet option sets — plus the defaults the dashboard itself uses. Values come from the same registries the dashboard renders its controls from, so this response is the authoritative input catalog for the other /api/v1/views endpoints. Inference charts for models with category deprecated append (Deprecated) to the source attribution, including PNG exports and unofficial-run overlays, without deprecated background watermarks. This is presentation only and does not change view data. No database access.
Parameters
| Name | Location | Type | Requirement | Description | Example |
|---|---|---|---|---|---|
| format | query | enum | Optional | Response encoding. This discovery endpoint is JSON-only. | json |
Request
curl
curl -sS 'https://inferencex.semianalysis.com/api/v1/views/options'Responses
ViewOptions200
Every option domain the views endpoints accept, plus dashboard-parity defaults. Each model includes deprecatedSequences, combining global and model-specific sequence retirements. Qwen3.5 8K/1K is deprecated; Qwen3.5 AgentX and explicit historical selections remain available.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"view": {
"type": "string",
"enum": [
"options"
]
},
"apiVersion": {
"type": "string",
"enum": [
"v1"
]
},
"params": {
"type": "object",
"properties": {
"format": {
"type": "string",
"enum": [
"json"
]
}
},
"required": [
"format"
],
"additionalProperties": false
},
"models": {
"type": "array",
"items": {
"type": "object",
"properties": {
"name": {
"type": "string"
},
"dbKeys": {
"type": "array",
"items": {
"type": "string"
}
},
"category": {
"type": "string"
},
"releaseDate": {
"type": [
"string",
"null"
]
},
"compareSlug": {
"type": [
"string",
"null"
]
},
"deprecatedSequences": {
"type": "array",
"items": {
"type": "string"
},
"description": "Sequence keys categorized as deprecated for this model, including globally deprecated sequences. Historical selections remain available; this does not retire the model."
}
},
"required": [
"name",
"dbKeys",
"category",
"releaseDate",
"compareSlug",
"deprecatedSequences"
],
"additionalProperties": false
},
"description": "Models in selector order within each category, with Kimi-K3.1-Pro first. The bare inference page opens Kimi-K3.1-Pro; this navigation default does not change defaults.model or remove the inference API requirement for an explicit model."
},
"sequences": {
"type": "array",
"items": {
"type": "object",
"properties": {
"key": {
"type": "string"
},
"label": {
"type": "string"
},
"labelZh": {
"type": "string"
},
"urlSegment": {
"type": [
"string",
"null"
]
},
"isl": {
"type": [
"integer",
"null"
]
},
"osl": {
"type": [
"integer",
"null"
]
},
"kind": {
"type": "string"
},
"deprecated": {
"type": "boolean"
}
},
"required": [
"key",
"label",
"labelZh",
"urlSegment",
"isl",
"osl",
"kind",
"deprecated"
],
"additionalProperties": false
}
},
"precisions": {
"type": "array",
"items": {
"type": "string"
}
},
"hardware": {
"type": "array",
"items": {
"type": "object",
"properties": {
"key": {
"type": "string"
},
"label": {
"type": "string"
},
"vendor": {
"type": "string"
},
"arch": {
"type": "string"
},
"tdpW": {
"type": "number",
"description": "Per-chip rated TDP in watts, shared with chip pages. B300 uses 1100 W."
},
"costPerHour": {
"type": "object",
"properties": {
"h": {
"type": "number"
},
"r": {
"type": "number"
}
},
"required": [
"h",
"r"
],
"additionalProperties": false
}
},
"required": [
"key",
"label",
"vendor",
"arch",
"tdpW",
"costPerHour"
],
"additionalProperties": false
}
},
"frameworks": {
"type": "array",
"items": {
"type": "object",
"properties": {
"key": {
"type": "string"
},
"label": {
"type": "string"
},
"family": {
"type": [
"string",
"null"
]
}
},
"required": [
"key",
"label",
"family"
],
"additionalProperties": false
}
},
"specMethods": {
"type": "array",
"items": {
"type": "string"
}
},
"percentiles": {
"type": "array",
"items": {
"type": "string"
}
},
"xAxisModes": {
"type": "array",
"items": {
"type": "string"
}
},
"fixedSequenceStatistics": {
"type": "array",
"items": {
"type": "string"
}
},
"scaleModes": {
"type": "array",
"items": {
"type": "string"
}
},
"metrics": {
"type": "array",
"items": {
"type": "object",
"properties": {
"key": {
"type": "string"
},
"configKey": {
"type": "string"
},
"label": {
"type": "string"
},
"labelZh": {
"type": "string"
},
"unit": {
"type": [
"string",
"null"
]
},
"polarity": {
"type": [
"string",
"null"
]
},
"group": {
"type": [
"string",
"null"
]
},
"source": {
"type": "string"
}
},
"required": [
"key",
"configKey",
"label",
"labelZh",
"unit",
"polarity",
"group",
"source"
],
"additionalProperties": false
}
},
"quickFilters": {
"type": "object",
"properties": {
"vendors": {
"type": "array",
"items": {
"type": "string"
}
},
"frameworkFamilies": {
"type": "array",
"items": {
"type": "string"
}
},
"deployments": {
"type": "array",
"items": {
"type": "string"
}
},
"specModes": {
"type": "array",
"items": {
"type": "string"
}
}
},
"required": [
"vendors",
"frameworkFamilies",
"deployments",
"specModes"
],
"additionalProperties": false
},
"reliabilityRanges": {
"type": "array",
"items": {
"type": "string"
}
},
"overview": {
"type": "object",
"additionalProperties": true
},
"calculator": {
"type": "object",
"additionalProperties": true
},
"fleet": {
"type": "object",
"additionalProperties": true
},
"defaults": {
"type": "object",
"additionalProperties": true
}
},
"required": [
"view",
"apiVersion",
"params",
"models",
"sequences",
"precisions",
"hardware",
"frameworks",
"metrics",
"defaults"
],
"additionalProperties": false
}Scroll to inspect the full example
Response example
{
"view": "options",
"apiVersion": "v1",
"params": {
"format": "json"
},
"models": [
{
"name": "Kimi-K3",
"dbKeys": [
"kimik3"
],
"category": "deprecated",
"releaseDate": "2026-07-27",
"compareSlug": "kimi-k3",
"deprecatedSequences": [
"1k/1k",
"1k/8k"
]
},
{
"name": "DeepSeek-V4-Pro",
"dbKeys": [
"dsv4"
],
"category": "deprecated",
"releaseDate": "2026-06-30",
"compareSlug": "deepseek-v4-pro",
"deprecatedSequences": [
"1k/1k",
"1k/8k",
"8k/1k"
]
},
{
"name": "MiniMax-M3",
"dbKeys": [
"minimaxm3"
],
"category": "deprecated",
"releaseDate": "2026-06-07",
"compareSlug": "minimax-m3",
"deprecatedSequences": [
"1k/1k",
"1k/8k",
"8k/1k"
]
}
],
"sequences": [
{
"key": "8k/1k",
"label": "8k/1k",
"labelZh": "8k/1k",
"urlSegment": "8k-1k",
"isl": 8192,
"osl": 1024,
"kind": "fixed",
"deprecated": false
}
],
"precisions": [
"fp4",
"fp4fp8",
"fp8",
"bf16",
"int4"
],
"hardware": [
{
"key": "h200",
"label": "H200",
"vendor": "NVIDIA",
"arch": "Hopper",
"tdpW": 700,
"costPerHour": {
"h": 2.29,
"n": 2.99,
"r": 3.11
}
}
],
"frameworks": [
{
"key": "trt",
"label": "TRTLLM",
"family": "trt"
}
],
"specMethods": [
"mtp",
"none"
],
"percentiles": [
"p75",
"p90"
],
"xAxisModes": [
"interactivity",
"ttft",
"e2e",
"e2e-normalized-interactivity"
],
"fixedSequenceStatistics": [
"median",
"mean"
],
"scaleModes": [
"auto",
"linear",
"log"
],
"metrics": [
{
"key": "tokensPerDollarN",
"configKey": "y_tokensPerDollarN",
"label": "Tokens per Dollar (Neocloud)",
"labelZh": "每美元 Token 数（Neocloud）",
"unit": null,
"polarity": "higher",
"group": "Cost",
"source": "benchmark"
}
],
"quickFilters": {
"vendors": [
"NVIDIA",
"AMD"
],
"frameworkFamilies": [
"vllm",
"sglang",
"trt",
"atom"
],
"deployments": [
"single-node",
"multi-node",
"disagg"
],
"specModes": [
"mtp",
"stp"
]
},
"reliabilityRanges": [
"last-3-days",
"last-7-days",
"last-month",
"last-3-months",
"all-time"
],
"defaults": {
"model": "DeepSeek-V4-Pro",
"sequence": "8k/1k",
"metric": "y_tokensPerDollarH",
"percentile": "p90",
"xmode": "interactivity",
"xstat": "median",
"serviceCompare": false,
"roleShare": false,
"powerFit": false,
"powerBreakdown": false,
"headroom": false
}
}Scroll to inspect the full example
400
The format value is invalid — this endpoint is JSON-only.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"error": {
"type": "string"
}
},
"required": [
"error"
],
"additionalProperties": true
}Response example
{
"error": "Unsupported format: csv",
"param": "format",
"allowed": [
"json"
]
}500
Option assembly failed.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"error": {
"type": "string"
}
},
"required": [
"error"
],
"additionalProperties": true
}Response example
{
"error": "Internal server error"
}GETBeta/api/v1/views/inferenceGet the main inference chart view
Returns the chart-ready series the /infx/inference scatter chart renders: per hardware config, x/y points at each measured concurrency for the selected metric, sequence, precisions and x-axis mode, with boundary and best-per-SKU flags computed by the same code the dashboard runs. Filters mirror the dashboard quick filters (gpus, vendors, framework families, deployment, spec). Use optimal=true for boundary points or best=true for the best series per GPU SKU. Measured-power boundaries follow the higher-power outer envelope: frontier.direction describes that boundary, while metric.direction remains the optimization direction used by best-per-SKU selection. All in Measured watts and energy support single_turn and AgentX rows with validated telemetry and supported topology; AgentX uses the power model’s agentic workload state without independent workload calibration. All in Measured adds tableRows with every GPU-valid observation in the selected scope and best-series selection (SKUs with no rankable estimate keep their rows), before optimal or axis clipping; unavailable estimates have y=null and a reason. series and count remain numeric chart points.
Parameters
| Name | Location | Type | Requirement | Description | Example |
|---|---|---|---|---|---|
| chipBasis | query | string | Optional | pool (default) or total. Chips that divide disaggregated input and output throughput and every metric derived from them: pool uses prefill chips for input and decode chips for output; total uses every chip in the deployment, the basis aggregated configs and tput_per_gpu always use. Aggregated rows are identical under both. | |
| topologies | query | string | Optional | Comma-separated exact topology keys from the Dashboard topology selector or view point topologyKey. Filters allocation and parallelism without filtering concurrency; omitted means all. Preserve the keys verbatim. Unknown keys return no matching points. | |
| allPoints | query | string | Optional | Boolean, default false. Include points clipped by dashboard limits; optimal and best still apply independently. | |
| best | query | boolean | Optional | Defaults to true: rank candidate frameworks per SKU within the selected precisions, using the current axes. Each candidate is scored by the area its measured Pareto frontier dominates (hypervolume from the origin, with lower-is-better axes taken as reciprocals so $/M tok ranks like tok/$); a curve earns nothing where it has no measurement. An unselected precision cannot displace a visible configuration. best=false returns all selected series, including vLLM and SGLang on the same SKU. Choosing a different model in the dashboard model selector resets manual legend choices to the destination model default; API requests are stateless, so pass best=false explicitly for each model when needed. The dashboard asks for a no-drama acknowledgement before showing both; the read-only API has no interactive consent state. This ranking is not a universal engine winner. | true |
| date | query | string | Optional | As-of date (YYYY-MM-DD): the latest run on or before this date per config. | 2026-08-01 |
| dates | query | string | Optional | Up to twelve comma-separated YYYY-MM-DD or YYYY-MM-DD~rRUN_ID comparison entries. Date-only entries select that exact logical snapshot, not an as-of cutoff; run entries select the exact logical run snapshot. Each snapshot is evaluated independently. | |
| deployment | query | string | Optional | Comma-separated deployment modes; agg expands to single-node and multi-node. | disagg |
| end | query | string | Optional | Comparison range end, YYYY-MM-DD; must not precede start. Historical uses an inclusive data bound. | |
| format | query | enum | Optional | Response encoding. csv returns plotted points, or tableRows for All in Measured with unavailable y cells blank. serviceCompare, roleShare, powerFit, powerBreakdown and headroom analytical results require JSON and return 400 with CSV. | csv |
| frameworks | query | string | Optional | Comma-separated framework families. | vllm,sglang |
| gpus | query | string | Optional | Comma-separated hardware keys or bare GPU names to include (e.g. h200 or a full hwKey). When omitted, Qwen3.5 excludes MI300X, MI325X, H100, and H200. An explicit list overrides this default; gpus= includes all hardware. | h200,mi355x |
| metric | query | string | Optional | Y-axis metric key or config key, e.g. tpPerGpu or y_tokensPerDollarN (see options → metrics). | y_tpPerGpu |
| model | query | string | Required | Frontend model display name (see /api/v1/views/options → models). | DeepSeek-V4-Pro |
| optimal | query | boolean | Optional | Return only boundary points per hardware, precision and snapshot date. Measured-power gauges use the higher-power outer envelope, matching the chart; other metrics use their Pareto frontier. The power envelope is not an efficiency recommendation. | true |
| percentile | query | enum | Optional | Latency percentile for agentic-trace x fields. | p75 |
| power | query | string | Optional | Comma-separated certified and/or legacy power tiers. Omit for all tiers. | |
| precisions | query | string | Optional | Comma-separated precisions. When omitted, the densest available precision is auto-selected, matching the dashboard default. | fp8,bf16 |
| priceSource | query | string | Optional | Profit: list, openrouter or custom. Inference/history: normalized or openrouter. Returned pricing records the prices actually used. | |
| runId | query | integer | Optional | GitHub Actions run id. Returns exactly that run snapshot instead of the latest data. | 12345678 |
| sequence | query | enum | Optional | Benchmark sequence. Aliases like 8k-1k and agentic are accepted. | agentic-traces |
| spec | query | string | Optional | Comma-separated speculative-decoding modes. | mtp |
| start | query | string | Optional | Comparison range start, YYYY-MM-DD. With end, adds the two endpoints, not every intermediate date. Historical uses an inclusive data bound. | |
| tcoBasis | query | string | Optional | internal (default) or external. Applies the same configured owning/rental cost basis as the UI. | |
| unofficialrun | query | string | Optional | Up to eight comma-separated public CI run IDs. Surrounding whitespace and duplicates are removed; run indices follow the remaining order. Overlay sources remain separate from official data. | |
| userCosts | query | string | Optional | JSON object from hardware keys to finite nonnegative USD/chip-hour values for user-priced metrics. | |
| userPowers | query | string | Optional | JSON object from hardware keys to finite nonnegative power assumptions in kW/chip. | |
| vendors | query | string | Optional | Comma-separated GPU vendors. | NVIDIA |
| xmetric | query | enum | Optional | TTFT percentile used when the x axis shows time to first token. | p99_ttft |
| xmode | query | enum | Optional | X-axis mode. Defaults to e2e for Qwen3.5 and interactivity for other models. An explicit value overrides the model default. e2e-normalized-interactivity uses persisted derived AgentX metrics, and unofficialrun overlays use the values reported by their run; points without eligible values are omitted. | e2e |
| xstat | query | enum | Optional | Fixed-sequence service-axis statistic: median (default) or mean. Mean streaming speed is 1 / mean TPOT, not the arithmetic mean of per-request speeds. Mean TTFT and E2E use their recorded mean fields. Missing values are omitted, never replaced with median. Ignored for AgentX; params.xstat then resolves to null and xAxis.statistic records the effective percentile. | mean |
| serviceCompare | query | boolean | Optional | API-only analysis: include source options, equal-service percentage curves, an optional target comparison and the same-concurrency diagnostic table (matchedConcurrency). Uses scoped observed points before frontier/best pruning. No corresponding dashboard control or panel. JSON only. | true |
| serviceBaseline | query | string | Optional | Exact baseline key from serviceSources. Omitted selects the first deterministic source; an unknown explicit key remains unavailable. Percent change is 100 × (comparator / baseline − 1). | Exact key returned in serviceSources |
| serviceComparator | query | string | Optional | Exact comparator key from serviceSources. Omitted selects the second deterministic source. Keys retain hardware, curve snapshot, recipe, image, topology and workload identity. Stitched telemetry producer/exporter hashes do not split a snapshot; rows without one retain their own run, date and hashes. Never substitute a hardware name. | Exact key returned in serviceSources |
| serviceTarget | query | number | Optional | Positive finite service-axis target: tok/s/user for streaming speed, seconds for TTFT/E2E. Omitted returns the curve and null target comparison. Numerical linear interpolation is bounded by each exact source; no extrapolation or interpolation across missing metric endpoints. Concurrency is unsupported. | 40 |
| roleShare | query | boolean | Optional | Include validated disaggregated prefill/decode energy shares on one output-token denominator, plus rolePoints with each pool’s W/GPU and role-local energy. Uses same-window aggregate J/output ÷ J/input to convert prefill J/input; share denominator is reconstructed prefill + decode energy, not pool-local token counts. Missing/invalid data is omitted or null. JSON only. | true |
| powerFit | query | boolean | Optional | Include one ordinary least-squares line per source: measured mean W/GPU = P0 + m × output tok/s per allocated GPU (disaggregated output spread over prefill and decode GPUs). P0 is the zero-output intercept, not measured idle power; m is marginal J/output token. Sources with fewer than 3 distinct output rates return observations with fit null. JSON only. | true |
| powerBreakdown | query | boolean | Optional | Include powerBreakdowns: per hardware line (hwKey + precision, as in the chart legend), each point’s All in Measured W/GPU split into measured GPU, measured Grace CPU (NVL72 with valid socket telemetry), and the modeled host CPU, memory, NICs, NVSwitch, storage, PCIe and misc parts, fans, PSU loss, scale-out network share and PUE facility overhead. Parts are the deployment share ÷ allocated GPUs and sum to totalWattsPerGpu. Points without validated GPU telemetry or a supported system model are omitted. JSON only. | true |
| headroom | query | boolean | Optional | Include headroom: the PowerX measured-vs-TDP table. Per hardware line (hwKey + precision), measured average and P90 W per chip, P90 as % of registry TDP and headroom = 100 − that, read at 25 / 50 / 100 / 300 tok/s/user on interactivityField by the chart’s own interpolation (throughput Pareto front, monotone Steffen spline). A cell outside a line’s measured range is null, never extrapolated; lines without P90 power, throughput, a registry TDP or two points are omitted. GPU board power only. JSON only. | true |
Request
curl
curl -sS 'https://inferencex.semianalysis.com/api/v1/views/inference?model=Kimi-K3&metric=y_tokensPerDollarN&sequence=agentic-traces'Responses
InferenceView200
Chart-ready series grouped by hardware config, with frontier and best-per-SKU flags per point.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"view": {
"type": "string",
"enum": [
"inference"
]
},
"apiVersion": {
"type": "string",
"enum": [
"v1"
]
},
"params": {
"type": "object",
"additionalProperties": true
},
"metric": {
"type": "object",
"properties": {
"key": {
"type": "string"
},
"configKey": {
"type": "string"
},
"label": {
"type": "string"
},
"labelZh": {
"type": "string"
},
"unit": {
"type": [
"string",
"null"
]
},
"polarity": {
"type": [
"string",
"null"
]
},
"direction": {
"type": [
"string",
"null"
],
"description": "Configured optimization direction, also used by best-per-SKU selection."
}
},
"required": [
"key",
"configKey",
"label",
"labelZh"
],
"additionalProperties": false
},
"xAxis": {
"type": "object",
"properties": {
"mode": {
"type": "string"
},
"field": {
"type": "string"
},
"label": {
"type": "string"
},
"statistic": {
"type": "string"
}
},
"required": [
"mode",
"field",
"label",
"statistic"
],
"additionalProperties": false
},
"frontier": {
"type": "object",
"properties": {
"direction": {
"type": [
"string",
"null"
],
"description": "Selected boundary direction. Measured-power gauges use upper_right for interactivity or upper_left for latency, independently of metric.direction."
},
"points": {
"type": "integer"
}
},
"required": [
"direction",
"points"
],
"additionalProperties": false
},
"hardware": {
"type": "array",
"items": {
"type": "object",
"properties": {
"key": {
"type": "string"
},
"label": {
"type": "string"
},
"vendor": {
"type": "string"
}
},
"required": [
"key",
"label"
],
"additionalProperties": false
}
},
"series": {
"type": "array",
"items": {
"type": "object",
"properties": {
"hwKey": {
"type": "string"
},
"gpu": {
"type": "string"
},
"framework": {
"type": "string"
},
"specMethod": {
"type": "string"
},
"label": {
"type": "string",
"description": "Chart display label. If any point in a plain SGLang/vLLM series has router_name=smg, the entire series is labeled SMG SGLang/SMG vLLM. Canonical framework and hwKey are unchanged."
},
"vendor": {
"type": "string"
},
"deployment": {
"type": "string"
},
"kvOffload": {
"type": "boolean"
},
"bestPerSku": {
"type": "boolean"
},
"points": {
"type": "array",
"items": {
"type": "object",
"properties": {
"id": {
"type": "integer"
},
"precision": {
"type": "string"
},
"x": {
"type": "number"
},
"y": {
"type": "number"
},
"concurrency": {
"type": "number"
},
"topologyKey": {
"type": "string"
},
"tp": {
"type": "number"
},
"date": {
"type": "string",
"format": "date"
},
"runId": {
"type": "integer"
},
"frontier": {
"type": "boolean"
},
"bestPerSku": {
"type": "boolean"
},
"metrics": {
"type": "object",
"additionalProperties": {
"type": "number"
}
}
},
"required": [
"x",
"y",
"concurrency",
"topologyKey",
"tp",
"date",
"frontier",
"bestPerSku",
"metrics"
],
"additionalProperties": false
}
}
},
"required": [
"hwKey",
"gpu",
"framework",
"specMethod",
"label",
"deployment",
"kvOffload",
"bestPerSku",
"points"
],
"additionalProperties": false
}
},
"tableRows": {
"type": "array",
"items": {
"type": "object",
"properties": {
"id": {
"type": [
"integer",
"null"
]
},
"precision": {
"type": "string"
},
"x": {
"type": [
"number",
"null"
],
"description": "Selected service axis; null when unavailable."
},
"y": {
"type": [
"number",
"null"
],
"description": "Selected all-in metric; null when unavailable."
},
"concurrency": {
"type": "number"
},
"topologyKey": {
"type": "string"
},
"tp": {
"type": "number"
},
"date": {
"type": "string",
"format": "date"
},
"runId": {
"type": "integer"
},
"frontier": {
"type": "boolean"
},
"bestPerSku": {
"type": "boolean"
},
"metrics": {
"type": "object",
"additionalProperties": {
"type": "number"
}
},
"hwKey": {
"type": "string"
},
"gpu": {
"type": "string"
},
"framework": {
"type": "string"
},
"specMethod": {
"type": "string"
},
"label": {
"type": "string",
"description": "Point-specific chart display label, including SMG SGLang/SMG vLLM when this point has router_name=smg. Canonical framework and hwKey are unchanged."
},
"vendor": {
"type": "string"
},
"deployment": {
"type": "string"
},
"kvOffload": {
"type": "boolean"
},
"measuredGpuWatts": {
"type": "number",
"description": "Validated measured mean W/GPU."
},
"status": {
"type": "string",
"enum": [
"available",
"unavailable"
]
},
"unavailableReason": {
"oneOf": [
{
"type": "string",
"enum": [
"workload",
"hardware",
"telemetry",
"gpu-count",
"topology",
"role-power",
"model-domain",
"energy",
"model-unavailable"
]
},
{
"type": "null"
}
]
}
},
"required": [
"id",
"precision",
"hwKey",
"gpu",
"framework",
"specMethod",
"label",
"deployment",
"kvOffload",
"x",
"y",
"concurrency",
"topologyKey",
"tp",
"date",
"frontier",
"bestPerSku",
"metrics",
"measuredGpuWatts",
"status",
"unavailableReason"
],
"additionalProperties": false
},
"description": "All in Measured only: GPU-valid observations in the selected scope, including unavailable estimates. Honors hardware/best selection; under best=true, SKUs with no rankable estimate keep their rows. Not clipped by optimal or chart axes. Also returned per comparison and overlay. count remains the numeric series point count."
},
"count": {
"type": "integer"
},
"serviceSources": {
"type": "array",
"items": {
"type": "object",
"properties": {
"key": {
"type": "string"
},
"label": {
"type": "string"
}
},
"required": [
"key",
"label"
],
"additionalProperties": false
}
},
"equalServiceComparison": {
"type": [
"object",
"null"
],
"properties": {
"target": {
"type": "number"
},
"xField": {
"type": "string"
},
"baseline": {
"type": [
"object",
"null"
],
"properties": {
"key": {
"type": "string"
},
"label": {
"type": "string"
}
},
"required": [
"key",
"label"
],
"additionalProperties": false
},
"comparator": {
"type": [
"object",
"null"
],
"properties": {
"key": {
"type": "string"
},
"label": {
"type": "string"
}
},
"required": [
"key",
"label"
],
"additionalProperties": false
},
"reason": {
"type": "string"
},
"metrics": {
"type": "object",
"properties": {
"meanWattsPerGpu": {
"type": "object",
"properties": {
"baseline": {
"type": [
"object",
"null"
],
"properties": {
"value": {
"type": "number"
},
"interpolated": {
"type": "boolean"
},
"endpoints": {
"type": "array",
"items": {
"type": "object",
"properties": {
"x": {
"type": "number"
},
"value": {
"type": "number"
},
"point": {
"type": "object",
"properties": {
"id": {
"type": [
"integer",
"null"
]
},
"sourceKey": {
"type": "string"
},
"hwKey": {
"type": "string"
},
"precision": {
"type": "string"
},
"concurrency": {
"type": "number"
},
"topologyKey": {
"type": "string"
},
"date": {
"type": "string"
},
"runUrl": {
"type": [
"string",
"null"
]
},
"recipeFingerprint": {
"type": [
"string",
"null"
]
},
"image": {
"type": [
"string",
"null"
]
}
},
"required": [
"id",
"sourceKey",
"hwKey",
"precision",
"concurrency",
"topologyKey",
"date",
"runUrl",
"recipeFingerprint",
"image"
],
"additionalProperties": false
}
},
"required": [
"x",
"value",
"point"
],
"additionalProperties": false
}
}
},
"required": [
"value",
"interpolated",
"endpoints"
],
"additionalProperties": false
},
"comparator": {
"type": [
"object",
"null"
],
"properties": {
"value": {
"type": "number"
},
"interpolated": {
"type": "boolean"
},
"endpoints": {
"type": "array",
"items": {
"type": "object",
"properties": {
"x": {
"type": "number"
},
"value": {
"type": "number"
},
"point": {
"type": "object",
"properties": {
"id": {
"type": [
"integer",
"null"
]
},
"sourceKey": {
"type": "string"
},
"hwKey": {
"type": "string"
},
"precision": {
"type": "string"
},
"concurrency": {
"type": "number"
},
"topologyKey": {
"type": "string"
},
"date": {
"type": "string"
},
"runUrl": {
"type": [
"string",
"null"
]
},
"recipeFingerprint": {
"type": [
"string",
"null"
]
},
"image": {
"type": [
"string",
"null"
]
}
},
"required": [
"id",
"sourceKey",
"hwKey",
"precision",
"concurrency",
"topologyKey",
"date",
"runUrl",
"recipeFingerprint",
"image"
],
"additionalProperties": false
}
},
"required": [
"x",
"value",
"point"
],
"additionalProperties": false
}
}
},
"required": [
"value",
"interpolated",
"endpoints"
],
"additionalProperties": false
},
"changePercent": {
"type": [
"number",
"null"
]
},
"reason": {
"type": "string"
}
},
"required": [
"baseline",
"comparator",
"changePercent",
"reason"
],
"additionalProperties": false,
"description": "Mean measured GPU board W/GPU."
},
"outputTokensPerSecond": {
"type": "object",
"properties": {
"baseline": {
"type": [
"object",
"null"
],
"properties": {
"value": {
"type": "number"
},
"interpolated": {
"type": "boolean"
},
"endpoints": {
"type": "array",
"items": {
"type": "object",
"properties": {
"x": {
"type": "number"
},
"value": {
"type": "number"
},
"point": {
"type": "object",
"properties": {
"id": {
"type": [
"integer",
"null"
]
},
"sourceKey": {
"type": "string"
},
"hwKey": {
"type": "string"
},
"precision": {
"type": "string"
},
"concurrency": {
"type": "number"
},
"topologyKey": {
"type": "string"
},
"date": {
"type": "string"
},
"runUrl": {
"type": [
"string",
"null"
]
},
"recipeFingerprint": {
"type": [
"string",
"null"
]
},
"image": {
"type": [
"string",
"null"
]
}
},
"required": [
"id",
"sourceKey",
"hwKey",
"precision",
"concurrency",
"topologyKey",
"date",
"runUrl",
"recipeFingerprint",
"image"
],
"additionalProperties": false
}
},
"required": [
"x",
"value",
"point"
],
"additionalProperties": false
}
}
},
"required": [
"value",
"interpolated",
"endpoints"
],
"additionalProperties": false
},
"comparator": {
"type": [
"object",
"null"
],
"properties": {
"value": {
"type": "number"
},
"interpolated": {
"type": "boolean"
},
"endpoints": {
"type": "array",
"items": {
"type": "object",
"properties": {
"x": {
"type": "number"
},
"value": {
"type": "number"
},
"point": {
"type": "object",
"properties": {
"id": {
"type": [
"integer",
"null"
]
},
"sourceKey": {
"type": "string"
},
"hwKey": {
"type": "string"
},
"precision": {
"type": "string"
},
"concurrency": {
"type": "number"
},
"topologyKey": {
"type": "string"
},
"date": {
"type": "string"
},
"runUrl": {
"type": [
"string",
"null"
]
},
"recipeFingerprint": {
"type": [
"string",
"null"
]
},
"image": {
"type": [
"string",
"null"
]
}
},
"required": [
"id",
"sourceKey",
"hwKey",
"precision",
"concurrency",
"topologyKey",
"date",
"runUrl",
"recipeFingerprint",
"image"
],
"additionalProperties": false
}
},
"required": [
"x",
"value",
"point"
],
"additionalProperties": false
}
}
},
"required": [
"value",
"interpolated",
"endpoints"
],
"additionalProperties": false
},
"changePercent": {
"type": [
"number",
"null"
]
},
"reason": {
"type": "string"
}
},
"required": [
"baseline",
"comparator",
"changePercent",
"reason"
],
"additionalProperties": false,
"description": "Whole-deployment output tokens/s; disaggregated GPU-count normalization preserves the workload denominator."
},
"joulesPerOutputToken": {
"type": "object",
"properties": {
"baseline": {
"type": [
"object",
"null"
],
"properties": {
"value": {
"type": "number"
},
"interpolated": {
"type": "boolean"
},
"endpoints": {
"type": "array",
"items": {
"type": "object",
"properties": {
"x": {
"type": "number"
},
"value": {
"type": "number"
},
"point": {
"type": "object",
"properties": {
"id": {
"type": [
"integer",
"null"
]
},
"sourceKey": {
"type": "string"
},
"hwKey": {
"type": "string"
},
"precision": {
"type": "string"
},
"concurrency": {
"type": "number"
},
"topologyKey": {
"type": "string"
},
"date": {
"type": "string"
},
"runUrl": {
"type": [
"string",
"null"
]
},
"recipeFingerprint": {
"type": [
"string",
"null"
]
},
"image": {
"type": [
"string",
"null"
]
}
},
"required": [
"id",
"sourceKey",
"hwKey",
"precision",
"concurrency",
"topologyKey",
"date",
"runUrl",
"recipeFingerprint",
"image"
],
"additionalProperties": false
}
},
"required": [
"x",
"value",
"point"
],
"additionalProperties": false
}
}
},
"required": [
"value",
"interpolated",
"endpoints"
],
"additionalProperties": false
},
"comparator": {
"type": [
"object",
"null"
],
"properties": {
"value": {
"type": "number"
},
"interpolated": {
"type": "boolean"
},
"endpoints": {
"type": "array",
"items": {
"type": "object",
"properties": {
"x": {
"type": "number"
},
"value": {
"type": "number"
},
"point": {
"type": "object",
"properties": {
"id": {
"type": [
"integer",
"null"
]
},
"sourceKey": {
"type": "string"
},
"hwKey": {
"type": "string"
},
"precision": {
"type": "string"
},
"concurrency": {
"type": "number"
},
"topologyKey": {
"type": "string"
},
"date": {
"type": "string"
},
"runUrl": {
"type": [
"string",
"null"
]
},
"recipeFingerprint": {
"type": [
"string",
"null"
]
},
"image": {
"type": [
"string",
"null"
]
}
},
"required": [
"id",
"sourceKey",
"hwKey",
"precision",
"concurrency",
"topologyKey",
"date",
"runUrl",
"recipeFingerprint",
"image"
],
"additionalProperties": false
}
},
"required": [
"x",
"value",
"point"
],
"additionalProperties": false
}
}
},
"required": [
"value",
"interpolated",
"endpoints"
],
"additionalProperties": false
},
"changePercent": {
"type": [
"number",
"null"
]
},
"reason": {
"type": "string"
}
},
"required": [
"baseline",
"comparator",
"changePercent",
"reason"
],
"additionalProperties": false,
"description": "Validated measured GPU joules per output token."
}
},
"required": [
"meanWattsPerGpu",
"outputTokensPerSecond",
"joulesPerOutputToken"
],
"additionalProperties": false
}
},
"required": [
"target",
"xField",
"baseline",
"comparator",
"reason",
"metrics"
],
"additionalProperties": false
},
"equalServiceCurve": {
"type": "array",
"items": {
"type": "object",
"properties": {
"target": {
"type": "number"
},
"xField": {
"type": "string"
},
"baseline": {
"type": [
"object",
"null"
],
"properties": {
"key": {
"type": "string"
},
"label": {
"type": "string"
}
},
"required": [
"key",
"label"
],
"additionalProperties": false
},
"comparator": {
"type": [
"object",
"null"
],
"properties": {
"key": {
"type": "string"
},
"label": {
"type": "string"
}
},
"required": [
"key",
"label"
],
"additionalProperties": false
},
"reason": {
"type": "string"
},
"metrics": {
"type": "object",
"properties": {
"meanWattsPerGpu": {
"type": "object",
"properties": {
"baseline": {
"type": [
"object",
"null"
],
"properties": {
"value": {
"type": "number"
},
"interpolated": {
"type": "boolean"
},
"endpoints": {
"type": "array",
"items": {
"type": "object",
"properties": {
"x": {
"type": "number"
},
"value": {
"type": "number"
},
"point": {
"type": "object",
"properties": {
"id": {
"type": [
"integer",
"null"
]
},
"sourceKey": {
"type": "string"
},
"hwKey": {
"type": "string"
},
"precision": {
"type": "string"
},
"concurrency": {
"type": "number"
},
"topologyKey": {
"type": "string"
},
"date": {
"type": "string"
},
"runUrl": {
"type": [
"string",
"null"
]
},
"recipeFingerprint": {
"type": [
"string",
"null"
]
},
"image": {
"type": [
"string",
"null"
]
}
},
"required": [
"id",
"sourceKey",
"hwKey",
"precision",
"concurrency",
"topologyKey",
"date",
"runUrl",
"recipeFingerprint",
"image"
],
"additionalProperties": false
}
},
"required": [
"x",
"value",
"point"
],
"additionalProperties": false
}
}
},
"required": [
"value",
"interpolated",
"endpoints"
],
"additionalProperties": false
},
"comparator": {
"type": [
"object",
"null"
],
"properties": {
"value": {
"type": "number"
},
"interpolated": {
"type": "boolean"
},
"endpoints": {
"type": "array",
"items": {
"type": "object",
"properties": {
"x": {
"type": "number"
},
"value": {
"type": "number"
},
"point": {
"type": "object",
"properties": {
"id": {
"type": [
"integer",
"null"
]
},
"sourceKey": {
"type": "string"
},
"hwKey": {
"type": "string"
},
"precision": {
"type": "string"
},
"concurrency": {
"type": "number"
},
"topologyKey": {
"type": "string"
},
"date": {
"type": "string"
},
"runUrl": {
"type": [
"string",
"null"
]
},
"recipeFingerprint": {
"type": [
"string",
"null"
]
},
"image": {
"type": [
"string",
"null"
]
}
},
"required": [
"id",
"sourceKey",
"hwKey",
"precision",
"concurrency",
"topologyKey",
"date",
"runUrl",
"recipeFingerprint",
"image"
],
"additionalProperties": false
}
},
"required": [
"x",
"value",
"point"
],
"additionalProperties": false
}
}
},
"required": [
"value",
"interpolated",
"endpoints"
],
"additionalProperties": false
},
"changePercent": {
"type": [
"number",
"null"
]
},
"reason": {
"type": "string"
}
},
"required": [
"baseline",
"comparator",
"changePercent",
"reason"
],
"additionalProperties": false,
"description": "Mean measured GPU board W/GPU."
},
"outputTokensPerSecond": {
"type": "object",
"properties": {
"baseline": {
"type": [
"object",
"null"
],
"properties": {
"value": {
"type": "number"
},
"interpolated": {
"type": "boolean"
},
"endpoints": {
"type": "array",
"items": {
"type": "object",
"properties": {
"x": {
"type": "number"
},
"value": {
"type": "number"
},
"point": {
"type": "object",
"properties": {
"id": {
"type": [
"integer",
"null"
]
},
"sourceKey": {
"type": "string"
},
"hwKey": {
"type": "string"
},
"precision": {
"type": "string"
},
"concurrency": {
"type": "number"
},
"topologyKey": {
"type": "string"
},
"date": {
"type": "string"
},
"runUrl": {
"type": [
"string",
"null"
]
},
"recipeFingerprint": {
"type": [
"string",
"null"
]
},
"image": {
"type": [
"string",
"null"
]
}
},
"required": [
"id",
"sourceKey",
"hwKey",
"precision",
"concurrency",
"topologyKey",
"date",
"runUrl",
"recipeFingerprint",
"image"
],
"additionalProperties": false
}
},
"required": [
"x",
"value",
"point"
],
"additionalProperties": false
}
}
},
"required": [
"value",
"interpolated",
"endpoints"
],
"additionalProperties": false
},
"comparator": {
"type": [
"object",
"null"
],
"properties": {
"value": {
"type": "number"
},
"interpolated": {
"type": "boolean"
},
"endpoints": {
"type": "array",
"items": {
"type": "object",
"properties": {
"x": {
"type": "number"
},
"value": {
"type": "number"
},
"point": {
"type": "object",
"properties": {
"id": {
"type": [
"integer",
"null"
]
},
"sourceKey": {
"type": "string"
},
"hwKey": {
"type": "string"
},
"precision": {
"type": "string"
},
"concurrency": {
"type": "number"
},
"topologyKey": {
"type": "string"
},
"date": {
"type": "string"
},
"runUrl": {
"type": [
"string",
"null"
]
},
"recipeFingerprint": {
"type": [
"string",
"null"
]
},
"image": {
"type": [
"string",
"null"
]
}
},
"required": [
"id",
"sourceKey",
"hwKey",
"precision",
"concurrency",
"topologyKey",
"date",
"runUrl",
"recipeFingerprint",
"image"
],
"additionalProperties": false
}
},
"required": [
"x",
"value",
"point"
],
"additionalProperties": false
}
}
},
"required": [
"value",
"interpolated",
"endpoints"
],
"additionalProperties": false
},
"changePercent": {
"type": [
"number",
"null"
]
},
"reason": {
"type": "string"
}
},
"required": [
"baseline",
"comparator",
"changePercent",
"reason"
],
"additionalProperties": false,
"description": "Whole-deployment output tokens/s; disaggregated GPU-count normalization preserves the workload denominator."
},
"joulesPerOutputToken": {
"type": "object",
"properties": {
"baseline": {
"type": [
"object",
"null"
],
"properties": {
"value": {
"type": "number"
},
"interpolated": {
"type": "boolean"
},
"endpoints": {
"type": "array",
"items": {
"type": "object",
"properties": {
"x": {
"type": "number"
},
"value": {
"type": "number"
},
"point": {
"type": "object",
"properties": {
"id": {
"type": [
"integer",
"null"
]
},
"sourceKey": {
"type": "string"
},
"hwKey": {
"type": "string"
},
"precision": {
"type": "string"
},
"concurrency": {
"type": "number"
},
"topologyKey": {
"type": "string"
},
"date": {
"type": "string"
},
"runUrl": {
"type": [
"string",
"null"
]
},
"recipeFingerprint": {
"type": [
"string",
"null"
]
},
"image": {
"type": [
"string",
"null"
]
}
},
"required": [
"id",
"sourceKey",
"hwKey",
"precision",
"concurrency",
"topologyKey",
"date",
"runUrl",
"recipeFingerprint",
"image"
],
"additionalProperties": false
}
},
"required": [
"x",
"value",
"point"
],
"additionalProperties": false
}
}
},
"required": [
"value",
"interpolated",
"endpoints"
],
"additionalProperties": false
},
"comparator": {
"type": [
"object",
"null"
],
"properties": {
"value": {
"type": "number"
},
"interpolated": {
"type": "boolean"
},
"endpoints": {
"type": "array",
"items": {
"type": "object",
"properties": {
"x": {
"type": "number"
},
"value": {
"type": "number"
},
"point": {
"type": "object",
"properties": {
"id": {
"type": [
"integer",
"null"
]
},
"sourceKey": {
"type": "string"
},
"hwKey": {
"type": "string"
},
"precision": {
"type": "string"
},
"concurrency": {
"type": "number"
},
"topologyKey": {
"type": "string"
},
"date": {
"type": "string"
},
"runUrl": {
"type": [
"string",
"null"
]
},
"recipeFingerprint": {
"type": [
"string",
"null"
]
},
"image": {
"type": [
"string",
"null"
]
}
},
"required": [
"id",
"sourceKey",
"hwKey",
"precision",
"concurrency",
"topologyKey",
"date",
"runUrl",
"recipeFingerprint",
"image"
],
"additionalProperties": false
}
},
"required": [
"x",
"value",
"point"
],
"additionalProperties": false
}
}
},
"required": [
"value",
"interpolated",
"endpoints"
],
"additionalProperties": false
},
"changePercent": {
"type": [
"number",
"null"
]
},
"reason": {
"type": "string"
}
},
"required": [
"baseline",
"comparator",
"changePercent",
"reason"
],
"additionalProperties": false,
"description": "Validated measured GPU joules per output token."
}
},
"required": [
"meanWattsPerGpu",
"outputTokensPerSecond",
"joulesPerOutputToken"
],
"additionalProperties": false
}
},
"required": [
"target",
"xField",
"baseline",
"comparator",
"reason",
"metrics"
],
"additionalProperties": false
}
},
"roleEnergyShares": {
"type": "array",
"items": {
"type": "object",
"properties": {
"x": {
"type": "number"
},
"sourceKey": {
"type": "string"
},
"point": {
"type": "object",
"properties": {
"id": {
"type": [
"integer",
"null"
]
},
"sourceKey": {
"type": "string"
},
"hwKey": {
"type": "string"
},
"precision": {
"type": "string"
},
"concurrency": {
"type": "number"
},
"topologyKey": {
"type": "string"
},
"date": {
"type": "string"
},
"runUrl": {
"type": [
"string",
"null"
]
},
"recipeFingerprint": {
"type": [
"string",
"null"
]
},
"image": {
"type": [
"string",
"null"
]
}
},
"required": [
"id",
"sourceKey",
"hwKey",
"precision",
"concurrency",
"topologyKey",
"date",
"runUrl",
"recipeFingerprint",
"image"
],
"additionalProperties": false
},
"prefill": {
"type": "number",
"description": "Prefill energy in J/output token."
},
"decode": {
"type": "number",
"description": "Decode energy in J/output token."
},
"total": {
"type": "number",
"description": "Reconstructed prefill + decode energy in J/output token."
},
"prefillShare": {
"type": "number",
"description": "Prefill percentage of reconstructed total."
},
"decodeShare": {
"type": "number",
"description": "Decode percentage of reconstructed total."
}
},
"required": [
"x",
"sourceKey",
"point",
"prefill",
"decode",
"total",
"prefillShare",
"decodeShare"
],
"additionalProperties": false
}
},
"matchedConcurrency": {
"type": "object",
"properties": {
"baseline": {
"type": [
"object",
"null"
],
"properties": {
"key": {
"type": "string"
},
"label": {
"type": "string"
}
},
"required": [
"key",
"label"
],
"additionalProperties": false
},
"comparator": {
"type": [
"object",
"null"
],
"properties": {
"key": {
"type": "string"
},
"label": {
"type": "string"
}
},
"required": [
"key",
"label"
],
"additionalProperties": false
},
"interactivityField": {
"type": "string"
},
"reason": {
"type": "string",
"enum": [
"same-source",
"unknown-source"
]
},
"rows": {
"type": "array",
"items": {
"type": "object",
"properties": {
"concurrency": {
"type": "integer"
},
"baseline": {
"type": "object",
"properties": {
"status": {
"type": "string",
"enum": [
"observed",
"missing",
"ambiguous"
],
"description": "observed: one agreeing observation; missing: none at this concurrency; ambiguous: conflicting observations, none chosen."
},
"values": {
"type": "object",
"properties": {
"joulesPerOutputToken": {
"type": [
"number",
"null"
]
},
"meanWattsPerGpu": {
"type": [
"number",
"null"
]
},
"interactivity": {
"type": [
"number",
"null"
]
}
},
"required": [
"joulesPerOutputToken",
"meanWattsPerGpu",
"interactivity"
],
"additionalProperties": false
},
"point": {
"type": "object",
"properties": {
"id": {
"type": [
"integer",
"null"
]
},
"sourceKey": {
"type": "string"
},
"hwKey": {
"type": "string"
},
"precision": {
"type": "string"
},
"concurrency": {
"type": "number"
},
"topologyKey": {
"type": "string"
},
"date": {
"type": "string"
},
"runUrl": {
"type": [
"string",
"null"
]
},
"recipeFingerprint": {
"type": [
"string",
"null"
]
},
"image": {
"type": [
"string",
"null"
]
}
},
"required": [
"id",
"sourceKey",
"hwKey",
"precision",
"concurrency",
"topologyKey",
"date",
"runUrl",
"recipeFingerprint",
"image"
],
"additionalProperties": false
},
"points": {
"type": "array",
"items": {
"type": "object",
"properties": {
"id": {
"type": [
"integer",
"null"
]
},
"sourceKey": {
"type": "string"
},
"hwKey": {
"type": "string"
},
"precision": {
"type": "string"
},
"concurrency": {
"type": "number"
},
"topologyKey": {
"type": "string"
},
"date": {
"type": "string"
},
"runUrl": {
"type": [
"string",
"null"
]
},
"recipeFingerprint": {
"type": [
"string",
"null"
]
},
"image": {
"type": [
"string",
"null"
]
}
},
"required": [
"id",
"sourceKey",
"hwKey",
"precision",
"concurrency",
"topologyKey",
"date",
"runUrl",
"recipeFingerprint",
"image"
],
"additionalProperties": false
}
}
},
"required": [
"status"
],
"additionalProperties": false
},
"comparator": {
"type": "object",
"properties": {
"status": {
"type": "string",
"enum": [
"observed",
"missing",
"ambiguous"
],
"description": "observed: one agreeing observation; missing: none at this concurrency; ambiguous: conflicting observations, none chosen."
},
"values": {
"type": "object",
"properties": {
"joulesPerOutputToken": {
"type": [
"number",
"null"
]
},
"meanWattsPerGpu": {
"type": [
"number",
"null"
]
},
"interactivity": {
"type": [
"number",
"null"
]
}
},
"required": [
"joulesPerOutputToken",
"meanWattsPerGpu",
"interactivity"
],
"additionalProperties": false
},
"point": {
"type": "object",
"properties": {
"id": {
"type": [
"integer",
"null"
]
},
"sourceKey": {
"type": "string"
},
"hwKey": {
"type": "string"
},
"precision": {
"type": "string"
},
"concurrency": {
"type": "number"
},
"topologyKey": {
"type": "string"
},
"date": {
"type": "string"
},
"runUrl": {
"type": [
"string",
"null"
]
},
"recipeFingerprint": {
"type": [
"string",
"null"
]
},
"image": {
"type": [
"string",
"null"
]
}
},
"required": [
"id",
"sourceKey",
"hwKey",
"precision",
"concurrency",
"topologyKey",
"date",
"runUrl",
"recipeFingerprint",
"image"
],
"additionalProperties": false
},
"points": {
"type": "array",
"items": {
"type": "object",
"properties": {
"id": {
"type": [
"integer",
"null"
]
},
"sourceKey": {
"type": "string"
},
"hwKey": {
"type": "string"
},
"precision": {
"type": "string"
},
"concurrency": {
"type": "number"
},
"topologyKey": {
"type": "string"
},
"date": {
"type": "string"
},
"runUrl": {
"type": [
"string",
"null"
]
},
"recipeFingerprint": {
"type": [
"string",
"null"
]
},
"image": {
"type": [
"string",
"null"
]
}
},
"required": [
"id",
"sourceKey",
"hwKey",
"precision",
"concurrency",
"topologyKey",
"date",
"runUrl",
"recipeFingerprint",
"image"
],
"additionalProperties": false
}
}
},
"required": [
"status"
],
"additionalProperties": false
},
"changePercent": {
"type": "object",
"properties": {
"joulesPerOutputToken": {
"type": [
"number",
"null"
]
},
"meanWattsPerGpu": {
"type": [
"number",
"null"
]
},
"interactivity": {
"type": [
"number",
"null"
]
}
},
"required": [
"joulesPerOutputToken",
"meanWattsPerGpu",
"interactivity"
],
"additionalProperties": false,
"description": "100 × (comparator / baseline − 1); null unless both values were observed."
}
},
"required": [
"concurrency",
"baseline",
"comparator",
"changePercent"
],
"additionalProperties": false
}
}
},
"required": [
"baseline",
"comparator",
"interactivityField",
"reason",
"rows"
],
"additionalProperties": false
},
"rolePoints": {
"type": "array",
"items": {
"type": "object",
"properties": {
"x": {
"type": "number"
},
"sourceKey": {
"type": "string"
},
"point": {
"type": "object",
"properties": {
"id": {
"type": [
"integer",
"null"
]
},
"sourceKey": {
"type": "string"
},
"hwKey": {
"type": "string"
},
"precision": {
"type": "string"
},
"concurrency": {
"type": "number"
},
"topologyKey": {
"type": "string"
},
"date": {
"type": "string"
},
"runUrl": {
"type": [
"string",
"null"
]
},
"recipeFingerprint": {
"type": [
"string",
"null"
]
},
"image": {
"type": [
"string",
"null"
]
}
},
"required": [
"id",
"sourceKey",
"hwKey",
"precision",
"concurrency",
"topologyKey",
"date",
"runUrl",
"recipeFingerprint",
"image"
],
"additionalProperties": false
},
"prefillWattsPerGpu": {
"type": [
"number",
"null"
],
"description": "Mean W/GPU inside the prefill pool."
},
"decodeWattsPerGpu": {
"type": [
"number",
"null"
],
"description": "Mean W/GPU inside the decode pool."
},
"prefillJoulesPerInputToken": {
"type": [
"number",
"null"
],
"description": "Prefill-pool joules per input token."
},
"decodeJoulesPerOutputToken": {
"type": [
"number",
"null"
],
"description": "Decode-pool joules per output token."
},
"energy": {
"type": [
"object",
"null"
],
"properties": {
"prefill": {
"type": "number"
},
"decode": {
"type": "number"
},
"total": {
"type": "number"
},
"prefillShare": {
"type": "number"
}
},
"required": [
"prefill",
"decode",
"total",
"prefillShare"
],
"additionalProperties": false,
"description": "Both pools on the output-token denominator (J/output token, share %)."
}
},
"required": [
"x",
"sourceKey",
"point",
"prefillWattsPerGpu",
"decodeWattsPerGpu",
"prefillJoulesPerInputToken",
"decodeJoulesPerOutputToken",
"energy"
],
"additionalProperties": false
}
},
"powerFits": {
"type": "array",
"items": {
"type": "object",
"properties": {
"source": {
"type": "object",
"properties": {
"key": {
"type": "string"
},
"label": {
"type": "string"
}
},
"required": [
"key",
"label"
],
"additionalProperties": false
},
"tdpWatts": {
"type": [
"number",
"null"
],
"description": "Rated board TDP per GPU; P0 ÷ TDP = fit.intercept / tdpWatts."
},
"fit": {
"type": [
"object",
"null"
],
"properties": {
"intercept": {
"type": "number",
"description": "P0, W/GPU at zero output (extrapolated)."
},
"slope": {
"type": "number",
"description": "m, J per output token."
},
"rSquared": {
"type": [
"number",
"null"
]
},
"n": {
"type": "integer"
},
"xMin": {
"type": "number"
},
"xMax": {
"type": "number"
}
},
"required": [
"intercept",
"slope",
"rSquared",
"n",
"xMin",
"xMax"
],
"additionalProperties": false
},
"reason": {
"type": "string",
"enum": [
"too-few-points"
]
},
"observations": {
"type": "array",
"items": {
"type": "object",
"properties": {
"x": {
"type": "number",
"description": "Output tok/s per allocated GPU."
},
"y": {
"type": "number",
"description": "Measured mean W/GPU."
},
"point": {
"type": "object",
"properties": {
"id": {
"type": [
"integer",
"null"
]
},
"sourceKey": {
"type": "string"
},
"hwKey": {
"type": "string"
},
"precision": {
"type": "string"
},
"concurrency": {
"type": "number"
},
"topologyKey": {
"type": "string"
},
"date": {
"type": "string"
},
"runUrl": {
"type": [
"string",
"null"
]
},
"recipeFingerprint": {
"type": [
"string",
"null"
]
},
"image": {
"type": [
"string",
"null"
]
}
},
"required": [
"id",
"sourceKey",
"hwKey",
"precision",
"concurrency",
"topologyKey",
"date",
"runUrl",
"recipeFingerprint",
"image"
],
"additionalProperties": false
}
},
"required": [
"x",
"y",
"point"
],
"additionalProperties": false
}
}
},
"required": [
"source",
"tdpWatts",
"fit",
"reason",
"observations"
],
"additionalProperties": false
}
},
"powerBreakdowns": {
"type": "array",
"items": {
"type": "object",
"properties": {
"key": {
"type": "string",
"description": "hwKey|precision."
},
"hwKey": {
"type": "string"
},
"precision": {
"type": "string"
},
"label": {
"type": "string",
"description": "Chart legend label, e.g. B200 (vLLM) FP4."
},
"categories": {
"type": "array",
"items": {
"type": "string",
"enum": [
"gpu",
"measuredCpu",
"cpu",
"memory",
"nic",
"nvswitch",
"storage",
"other",
"fans",
"psuLoss",
"network",
"facility"
],
"description": "Parts with watts on at least one point, in stack order. gpu and measuredCpu are measured; the rest are modeled."
}
},
"tdpWattsPerGpu": {
"type": [
"number",
"null"
],
"description": "Rated board TDP per GPU (reference, not a measurement)."
},
"allInProvisionedWattsPerGpu": {
"type": [
"number",
"null"
],
"description": "Registry all-in provisioned W/GPU, PUE included (reference, not a measurement)."
},
"modelRevision": {
"type": "string",
"description": "Pinned power_model commit."
},
"interactivityField": {
"type": "string",
"description": "Interactivity field the points are placed on: the chart x-axis when it is an interactivity axis, otherwise median_intvty."
},
"points": {
"type": "array",
"items": {
"type": "object",
"properties": {
"concurrency": {
"type": "integer"
},
"interactivity": {
"type": "number",
"description": "tok/s/user on interactivityField; points are sorted by it."
},
"configLabel": {
"type": "string",
"description": "Date, topology and recipe of the point."
},
"wattsPerGpu": {
"type": "object",
"properties": {
"gpu": {
"type": "number"
},
"measuredCpu": {
"type": "number"
},
"cpu": {
"type": "number"
},
"memory": {
"type": "number"
},
"nic": {
"type": "number"
},
"nvswitch": {
"type": "number"
},
"storage": {
"type": "number"
},
"other": {
"type": "number"
},
"fans": {
"type": "number"
},
"psuLoss": {
"type": "number"
},
"network": {
"type": "number"
},
"facility": {
"type": "number"
}
},
"required": [
"gpu",
"measuredCpu",
"cpu",
"memory",
"nic",
"nvswitch",
"storage",
"other",
"fans",
"psuLoss",
"network",
"facility"
],
"additionalProperties": false
},
"totalWattsPerGpu": {
"type": "number",
"description": "All in Measured W/GPU."
},
"pue": {
"type": "number"
},
"cpuBasis": {
"type": [
"string",
"null"
],
"description": "`measured` or `modeled` Grace sockets on NVL72 trays; null otherwise."
},
"chassisBasis": {
"type": "string",
"enum": [
"full",
"extrapolated"
]
},
"point": {
"type": "object",
"properties": {
"id": {
"type": [
"integer",
"null"
]
},
"sourceKey": {
"type": "string"
},
"hwKey": {
"type": "string"
},
"precision": {
"type": "string"
},
"concurrency": {
"type": "number"
},
"topologyKey": {
"type": "string"
},
"date": {
"type": "string"
},
"runUrl": {
"type": [
"string",
"null"
]
},
"recipeFingerprint": {
"type": [
"string",
"null"
]
},
"image": {
"type": [
"string",
"null"
]
}
},
"required": [
"id",
"sourceKey",
"hwKey",
"precision",
"concurrency",
"topologyKey",
"date",
"runUrl",
"recipeFingerprint",
"image"
],
"additionalProperties": false
}
},
"required": [
"concurrency",
"interactivity",
"configLabel",
"wattsPerGpu",
"totalWattsPerGpu",
"pue",
"cpuBasis",
"chassisBasis",
"point"
],
"additionalProperties": false
}
}
},
"required": [
"key",
"hwKey",
"precision",
"label",
"categories",
"tdpWattsPerGpu",
"allInProvisionedWattsPerGpu",
"modelRevision",
"interactivityField",
"points"
],
"additionalProperties": false
}
},
"headroom": {
"type": "object",
"properties": {
"interactivities": {
"type": "array",
"items": {
"type": "number"
}
},
"interactivityField": {
"type": "string",
"description": "Interactivity field the cells are read on: the chart x-axis when it is an interactivity axis, otherwise median_intvty."
},
"rows": {
"type": "array",
"items": {
"type": "object",
"properties": {
"key": {
"type": "string",
"description": "hwKey|precision."
},
"hwKey": {
"type": "string"
},
"precision": {
"type": "string"
},
"label": {
"type": "string",
"description": "Chart legend label, e.g. B200 (vLLM) FP4."
},
"tdpWatts": {
"type": "number",
"description": "Rated board TDP per GPU from the registry."
},
"interactivityMin": {
"type": "number"
},
"interactivityMax": {
"type": "number"
},
"cells": {
"type": "array",
"items": {
"oneOf": [
{
"type": "object",
"properties": {
"interactivity": {
"type": "number"
},
"avgWatts": {
"type": "number",
"description": "Interpolated measured average W per chip."
},
"p90Watts": {
"type": "number",
"description": "Interpolated measured P90 fleet W per chip."
},
"percentTdpP90": {
"type": "number",
"description": "P90 ÷ TDP × 100."
},
"headroomPercent": {
"type": "number",
"description": "100 − percentTdpP90."
}
},
"required": [
"interactivity",
"avgWatts",
"p90Watts",
"percentTdpP90",
"headroomPercent"
],
"additionalProperties": false
},
{
"type": "null"
}
]
}
}
},
"required": [
"key",
"hwKey",
"precision",
"label",
"tdpWatts",
"interactivityMin",
"interactivityMax",
"cells"
],
"additionalProperties": false
}
}
},
"required": [
"interactivities",
"interactivityField",
"rows"
],
"additionalProperties": false
},
"pricing": {
"type": [
"object",
"null"
],
"additionalProperties": true
},
"comparisons": {
"type": "array",
"items": {
"type": "object",
"properties": {
"tableRows": {
"type": "array",
"items": {
"type": "object",
"properties": {
"id": {
"type": [
"integer",
"null"
]
},
"precision": {
"type": "string"
},
"x": {
"type": [
"number",
"null"
],
"description": "Selected service axis; null when unavailable."
},
"y": {
"type": [
"number",
"null"
],
"description": "Selected all-in metric; null when unavailable."
},
"concurrency": {
"type": "number"
},
"topologyKey": {
"type": "string"
},
"tp": {
"type": "number"
},
"date": {
"type": "string",
"format": "date"
},
"runId": {
"type": "integer"
},
"frontier": {
"type": "boolean"
},
"bestPerSku": {
"type": "boolean"
},
"metrics": {
"type": "object",
"additionalProperties": {
"type": "number"
}
},
"hwKey": {
"type": "string"
},
"gpu": {
"type": "string"
},
"framework": {
"type": "string"
},
"specMethod": {
"type": "string"
},
"label": {
"type": "string",
"description": "Point-specific chart display label, including SMG SGLang/SMG vLLM when this point has router_name=smg. Canonical framework and hwKey are unchanged."
},
"vendor": {
"type": "string"
},
"deployment": {
"type": "string"
},
"kvOffload": {
"type": "boolean"
},
"measuredGpuWatts": {
"type": "number",
"description": "Validated measured mean W/GPU."
},
"status": {
"type": "string",
"enum": [
"available",
"unavailable"
]
},
"unavailableReason": {
"oneOf": [
{
"type": "string",
"enum": [
"workload",
"hardware",
"telemetry",
"gpu-count",
"topology",
"role-power",
"model-domain",
"energy",
"model-unavailable"
]
},
{
"type": "null"
}
]
}
},
"required": [
"id",
"precision",
"hwKey",
"gpu",
"framework",
"specMethod",
"label",
"deployment",
"kvOffload",
"x",
"y",
"concurrency",
"topologyKey",
"tp",
"date",
"frontier",
"bestPerSku",
"metrics",
"measuredGpuWatts",
"status",
"unavailableReason"
],
"additionalProperties": false
}
}
},
"additionalProperties": true
}
},
"overlays": {
"type": "array",
"items": {
"type": "object",
"properties": {
"tableRows": {
"type": "array",
"items": {
"type": "object",
"properties": {
"id": {
"type": [
"integer",
"null"
]
},
"precision": {
"type": "string"
},
"x": {
"type": [
"number",
"null"
],
"description": "Selected service axis; null when unavailable."
},
"y": {
"type": [
"number",
"null"
],
"description": "Selected all-in metric; null when unavailable."
},
"concurrency": {
"type": "number"
},
"topologyKey": {
"type": "string"
},
"tp": {
"type": "number"
},
"date": {
"type": "string",
"format": "date"
},
"runId": {
"type": "integer"
},
"frontier": {
"type": "boolean"
},
"bestPerSku": {
"type": "boolean"
},
"metrics": {
"type": "object",
"additionalProperties": {
"type": "number"
}
},
"hwKey": {
"type": "string"
},
"gpu": {
"type": "string"
},
"framework": {
"type": "string"
},
"specMethod": {
"type": "string"
},
"label": {
"type": "string",
"description": "Point-specific chart display label, including SMG SGLang/SMG vLLM when this point has router_name=smg. Canonical framework and hwKey are unchanged."
},
"vendor": {
"type": "string"
},
"deployment": {
"type": "string"
},
"kvOffload": {
"type": "boolean"
},
"measuredGpuWatts": {
"type": "number",
"description": "Validated measured mean W/GPU."
},
"status": {
"type": "string",
"enum": [
"available",
"unavailable"
]
},
"unavailableReason": {
"oneOf": [
{
"type": "string",
"enum": [
"workload",
"hardware",
"telemetry",
"gpu-count",
"topology",
"role-power",
"model-domain",
"energy",
"model-unavailable"
]
},
{
"type": "null"
}
]
}
},
"required": [
"id",
"precision",
"hwKey",
"gpu",
"framework",
"specMethod",
"label",
"deployment",
"kvOffload",
"x",
"y",
"concurrency",
"topologyKey",
"tp",
"date",
"frontier",
"bestPerSku",
"metrics",
"measuredGpuWatts",
"status",
"unavailableReason"
],
"additionalProperties": false
}
}
},
"additionalProperties": true
}
}
},
"required": [
"view",
"apiVersion",
"params",
"metric",
"xAxis",
"frontier",
"series",
"count"
],
"additionalProperties": false
}Scroll to inspect the full example
Response example
{
"view": "inference",
"apiVersion": "v1",
"params": {
"model": "DeepSeek-V4-Pro",
"sequence": "8k/1k",
"precisions": [
"fp8"
],
"metric": "y_tpPerGpu",
"xmode": "interactivity",
"xstat": "median",
"xmetric": "p90_ttft",
"percentile": "p90",
"date": null,
"runId": null,
"gpus": [],
"vendors": [],
"frameworks": [],
"deployment": [],
"spec": [],
"topologies": [],
"optimal": true,
"best": true,
"format": "json",
"serviceCompare": false,
"serviceBaseline": null,
"serviceComparator": null,
"serviceTarget": null,
"roleShare": false,
"powerFit": false,
"powerBreakdown": false,
"headroom": false
},
"metric": {
"key": "tpPerGpu",
"configKey": "y_tpPerGpu",
"label": "Output Throughput per GPU (tok/s/gpu)",
"labelZh": "单 GPU 输出吞吐（tok/s/gpu）",
"unit": "tok/s/gpu",
"polarity": "higher",
"direction": "upper_left"
},
"xAxis": {
"mode": "interactivity",
"field": "median_intvty",
"statistic": "median",
"label": "Median Interactivity (tok/s/user)"
},
"frontier": {
"direction": "upper_left",
"points": 14
},
"hardware": [
{
"key": "h200_trt",
"label": "H200 (TRTLLM)",
"vendor": "NVIDIA"
}
],
"series": [
{
"hwKey": "h200_trt",
"gpu": "h200",
"framework": "trt",
"specMethod": "none",
"label": "H200 (TRTLLM)",
"vendor": "NVIDIA",
"deployment": "single-node",
"kvOffload": false,
"bestPerSku": true,
"points": [
{
"x": 12.5,
"y": 450.5,
"concurrency": 64,
"topologyKey": "Single|GPU=8|DP=?|TP=8|EP=1|PP=?|DCP=?|PCP=?|DPA=0|offload=off",
"tp": 8,
"date": "2026-08-20",
"runId": 12345678,
"frontier": true,
"bestPerSku": true,
"metrics": {
"tpPerGpu": 450.5,
"outputTputPerGpu": 400.2,
"inputTputPerGpu": 50.3
}
}
]
}
],
"count": 1
}Scroll to inspect the full example
Alternate representation: text/csv
Response shape
{
"type": "string"
}Response example
hwKey,gpu,framework,specMethod,label,vendor,deployment,kvOffload,x,y,concurrency,topologyKey,tp,date,runId,frontier,bestPerSku,metric_tpPerGpu
h200_trt,h200,trt,none,H200 (TRTLLM),NVIDIA,single-node,false,12.5,450.5,64,Single|GPU=8|DP=?|TP=8|EP=1|PP=?|DCP=?|PCP=?|DPA=0|offload=off,8,2026-08-20,12345678,true,true,450.5400
A parameter is invalid. The body names the parameter and, for enums, lists the allowed values.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"error": {
"type": "string"
}
},
"required": [
"error"
],
"additionalProperties": true
}Response example
{
"error": "Unknown xmode: bogus",
"param": "xmode",
"allowed": [
"interactivity",
"ttft",
"e2e",
"e2e-normalized-interactivity"
]
}500
The benchmark query failed.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"error": {
"type": "string"
}
},
"required": [
"error"
],
"additionalProperties": true
}Response example
{
"error": "Internal server error"
}GETBeta/api/v1/views/historicalGet the historical trends view
Returns the Historical Trends dashboard lines computed server-side: for every benchmark snapshot date, the selected metric is interpolated at the target interactivity per hardware config (per precision when several are selected), then assembled into date-sorted trend lines. Interpolation uses the same monotone-spline math as the dashboard. Lines are extended to extendToDate (current UTC date by default), using a synthetic copy of their last value marked synthetic: true. Pass extendToDate explicitly for reproducible snapshots.
Parameters
| Name | Location | Type | Requirement | Description | Example |
|---|---|---|---|---|---|
| chipBasis | query | string | Optional | pool (default) or total. Chips that divide disaggregated input and output throughput and every metric derived from them: pool uses prefill chips for input and decode chips for output; total uses every chip in the deployment, the basis aggregated configs and tput_per_gpu always use. Aggregated rows are identical under both. | |
| chart | query | enum | Optional | line returns the original trend series. waterfall also returns baseline, record-setting changes, and best-observed steps with source runs and strictly matched PR links. Synthetic extensions and recoveries are excluded. Output-only metrics exclude curves without recorded output throughput; excludedPointCount reports omitted dates. Lower-is-better metrics use descending changes. Links identify source curves, not isolated causal effects. | waterfall |
| deployment | query | string | Optional | Comma-separated deployment modes; agg expands to single-node and multi-node. | single-node |
| end | query | string | Optional | Latest snapshot date to include (YYYY-MM-DD). | 2026-08-01 |
| extendToDate | query | string | Optional | Synthetic history-line end date, YYYY-MM-DD; defaults to current UTC date, matching the dashboard. | |
| format | query | enum | Optional | Response encoding. csv returns one row per line point or waterfall step, with targetInteractivity, interactivityStatistic and interactivityField. Waterfall rows include JSON-encoded runUrls and prLinks arrays. | csv |
| frameworks | query | string | Optional | Comma-separated framework families. | vllm,trt |
| gpus | query | string | Optional | Comma-separated hardware keys or bare GPU names to include. | h200,mi355x |
| metric | query | string | Optional | Trend metric key or config key, e.g. tpPerGpu or y_tokensPerDollarN (see options → metrics). | y_tpPerGpu |
| model | query | string | Required | Frontend model display name (see /api/v1/views/options → models). | DeepSeek-V4-Pro |
| precisions | query | string | Optional | Comma-separated precisions. When omitted, the densest available precision is auto-selected. With multiple precisions, each hardware gets one line per precision (key hwKey__precision). | fp8,bf16 |
| priceSource | query | string | Optional | Profit: list, openrouter or custom. Inference/history: normalized or openrouter. Returned pricing records the prices actually used. | |
| sequence | query | enum | Optional | Benchmark sequence. Aliases like 8k-1k and agentic are accepted. | agentic-traces |
| start | query | string | Optional | Earliest snapshot date to include (YYYY-MM-DD). | 2026-01-01 |
| target | query | number | Optional | Target interactivity (tok/s/user) at which the metric is interpolated for every snapshot. AgentX uses P90 slow-tail interactivity: 1 / P90 full-response ITL in seconds, with legacy ITL only when full-response mean/percentile ITL fields are absent. Missing or invalid P90 latency is omitted, never replaced with a median or raw rate percentile. Single-turn workloads retain median interactivity. | 50 |
| tcoBasis | query | string | Optional | internal (default) or external. Applies the same configured owning/rental cost basis as the UI. | |
| vendors | query | string | Optional | Comma-separated GPU vendors. | AMD |
Request
curl
curl -sS 'https://inferencex.semianalysis.com/api/v1/views/historical?model=DeepSeek-V4-Pro&metric=y_tpPerGpu&target=35'Responses
HistoricalView200
Date-sorted trend lines per hardware config, interpolated at the target interactivity.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"view": {
"type": "string",
"enum": [
"historical"
]
},
"apiVersion": {
"type": "string",
"enum": [
"v1"
]
},
"params": {
"type": "object",
"additionalProperties": true
},
"metric": {
"type": "object",
"properties": {
"key": {
"type": "string"
},
"configKey": {
"type": "string"
},
"label": {
"type": "string"
},
"labelZh": {
"type": "string"
}
},
"required": [
"key",
"configKey",
"label",
"labelZh"
],
"additionalProperties": false
},
"target": {
"type": "number"
},
"interactivity": {
"type": "object",
"properties": {
"statistic": {
"type": "string",
"enum": [
"median",
"p90"
]
},
"field": {
"type": "string",
"enum": [
"median_intvty",
"p90_intvty"
]
},
"unit": {
"type": "string",
"enum": [
"tok/s/user"
]
}
},
"required": [
"statistic",
"field",
"unit"
],
"additionalProperties": false
},
"hwKeysWithData": {
"type": "array",
"items": {
"type": "string"
}
},
"series": {
"type": "array",
"items": {
"type": "object",
"properties": {
"key": {
"type": "string"
},
"hwKey": {
"type": "string"
},
"precision": {
"type": [
"string",
"null"
]
},
"label": {
"type": "string"
},
"vendor": {
"type": [
"string",
"null"
]
},
"points": {
"type": "array",
"items": {
"type": "object",
"properties": {
"date": {
"type": "string",
"format": "date"
},
"value": {
"type": "number"
},
"synthetic": {
"type": "boolean"
}
},
"required": [
"date",
"value"
],
"additionalProperties": false
}
},
"steps": {
"type": "array",
"items": {
"type": "object",
"properties": {
"kind": {
"type": "string",
"enum": [
"baseline",
"gain",
"best"
]
},
"date": {
"type": "string"
},
"from": {
"type": "number"
},
"to": {
"type": "number"
},
"delta": {
"type": "number"
},
"sources": {
"type": "array",
"items": {
"type": "object",
"properties": {
"date": {
"type": "string"
},
"runId": {
"type": "integer"
},
"model": {
"type": "string"
},
"precision": {
"type": "string"
},
"hwKey": {
"type": "string"
},
"benchmarkType": {
"type": "string"
},
"runUrl": {
"type": "string"
}
},
"required": [
"date",
"runId",
"model",
"precision",
"hwKey",
"benchmarkType",
"runUrl"
],
"additionalProperties": false
}
},
"prLinks": {
"type": "array",
"items": {
"type": "string"
}
},
"linksAvailable": {
"type": "boolean"
}
},
"required": [
"kind",
"date",
"from",
"to",
"delta",
"sources",
"prLinks",
"linksAvailable"
],
"additionalProperties": false
}
},
"excludedPointCount": {
"type": "integer"
}
},
"required": [
"key",
"hwKey",
"label",
"points"
],
"additionalProperties": false
}
},
"count": {
"type": "integer"
}
},
"required": [
"view",
"apiVersion",
"params",
"metric",
"target",
"interactivity",
"hwKeysWithData",
"series",
"count"
],
"additionalProperties": false
}Scroll to inspect the full example
Response example
{
"view": "historical",
"apiVersion": "v1",
"params": {
"model": "DeepSeek-V4-Pro",
"sequence": "8k/1k",
"metric": "y_tpPerGpu",
"target": 35,
"precisions": [
"fp8"
],
"gpus": [],
"vendors": [],
"frameworks": [],
"deployment": [],
"start": null,
"end": null,
"format": "json"
},
"metric": {
"key": "tpPerGpu",
"configKey": "y_tpPerGpu",
"label": "Output Throughput per GPU (tok/s/gpu)",
"labelZh": "单 GPU 输出吞吐（tok/s/gpu）"
},
"target": 35,
"interactivity": {
"statistic": "median",
"field": "median_intvty",
"unit": "tok/s/user"
},
"hwKeysWithData": [
"h200_trt"
],
"series": [
{
"key": "h200_trt",
"hwKey": "h200_trt",
"precision": null,
"label": "H200 (TRTLLM)",
"vendor": "NVIDIA",
"points": [
{
"date": "2026-07-01",
"value": 310.4
},
{
"date": "2026-08-20",
"value": 355.2
},
{
"date": "2026-08-25",
"value": 355.2,
"synthetic": true
}
]
}
],
"count": 3
}Scroll to inspect the full example
Alternate representation: text/csv
Response shape
{
"type": "string"
}Response example
key,hwKey,precision,label,vendor,date,value,synthetic,targetInteractivity,interactivityStatistic,interactivityField
h200_trt,h200_trt,,H200 (TRTLLM),NVIDIA,2026-07-01,310.4,false,35,median,median_intvty400
A parameter is invalid. The body names the parameter and, for enums, lists the allowed values.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"error": {
"type": "string"
}
},
"required": [
"error"
],
"additionalProperties": true
}Response example
{
"error": "target must be >= 1",
"param": "target"
}500
The benchmark history query failed.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"error": {
"type": "string"
}
},
"required": [
"error"
],
"additionalProperties": true
}Response example
{
"error": "Internal server error"
}GETBeta/api/v1/views/calculatorInterpolated calculator operating points
Computes the throughput calculator server-side: benchmark sweeps are grouped per hardware config, reduced to their Pareto frontier, and read at the requested target with the same monotone Steffen–Hermite interpolation the dashboard uses. Optionally sizes a fixed-power fleet per config (mw) and reports each config’s maximum interactivity under a $/M-token cost cap (costcap).
Parameters
| Name | Location | Type | Requirement | Description | Example |
|---|---|---|---|---|---|
| chipBasis | query | string | Optional | pool (default) or total. Chips that divide disaggregated input and output throughput and every metric derived from them: pool uses prefill chips for input and decode chips for output; total uses every chip in the deployment, the basis aggregated configs and tput_per_gpu always use. Aggregated rows are identical under both. | |
| costProvider | query | enum | Optional | Cost basis: costh (owning) or costr (renting), with tcoBasis selecting internal or external assumptions. | costh |
| costType | query | enum | Optional | Token basis for cost and throughput fields. | total |
| costcap | query | number | Optional | Cost cap in $/M tokens. When set, the response carries a costCap section with each config’s maximum interactivity under the cap. | 2.5 |
| date | query | date | Optional | Use data on or before YYYY-MM-DD. Omit for latest. | 2026-08-08 |
| format | query | enum | Optional | Response encoding. | json |
| gpus | query | CSV list | Optional | Comma-separated hardware keys; matches a full hwKey (b300_sglang) or a base chip (b300). | b300,mi355x |
| hideSkuAboveConfigLimit | query | string | Optional | Boolean, default true. Omit hardware whose interpolation clamps above its measured configuration limit. | |
| mode | query | enum | Optional | Interpolation direction. | interactivity-to-throughput |
| model | query | string | Required | Display model name or compare slug. | DeepSeek-V4-Pro |
| mw | query | number | Optional | Facility power budget in MW. When set, each result carries a fleet block (chips, total tok/s, concurrent users, cost). | 100 |
| percentile | query | enum | Optional | Interactivity percentile for agentic traces; fixed sequences use the median. | p90 |
| precisions | query | CSV list | Optional | Comma-separated precisions. Omit to auto-select the densest official precision and include precisions present in unofficial-run overlays. | fp4,fp8 |
| runId | query | string | Optional | Numeric GitHub workflow run id to pin the snapshot to. | 123456789 |
| sequence | query | enum | Optional | Workload sequence. Accepts 1k/1k, 1k/8k, 8k/1k (or hyphenated forms) and agentic-traces (alias agentic). | 1k/1k |
| target | query | number | Optional | Target value on the input axis: interactivity in tok/s/user, or throughput in tok/s/GPU when mode is throughput-to-interactivity. | 35 |
| tcoBasis | query | string | Optional | internal (default) or external. Applies the same configured owning/rental cost basis as the UI. | |
| unofficialrun | query | string | Optional | Up to eight comma-separated public CI run IDs. Surrounding whitespace and duplicates are removed; run indices follow the remaining order. Overlay sources remain separate from official data. |
Request
curl
curl -sS 'https://inferencex.semianalysis.com/api/v1/views/calculator?model=DeepSeek-V4-Pro&sequence=1k/1k'Responses
CalculatorView200
Steffen–Hermite interpolated operating point per hardware config at the requested target.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"view": {
"type": "string",
"enum": [
"calculator"
]
},
"apiVersion": {
"type": "string",
"enum": [
"v1"
]
},
"generatedAt": {
"type": [
"string",
"null"
],
"description": "Latest run date among the rows the view was computed from."
},
"params": {
"type": "object",
"additionalProperties": true
},
"hardware": {
"type": "array",
"items": {
"type": "object",
"properties": {
"hwKey": {
"type": "string"
},
"resultKey": {
"type": "string"
},
"source": {
"type": "string"
},
"precision": {
"type": [
"string",
"null"
]
},
"label": {
"type": "string"
},
"value": {
"type": "number"
},
"inputThroughput": {
"type": "number"
},
"outputThroughput": {
"type": "number"
},
"cost": {
"type": "object",
"properties": {
"total": {
"type": "number"
},
"input": {
"type": "number"
},
"output": {
"type": "number"
}
},
"required": [
"total",
"input",
"output"
],
"additionalProperties": false
},
"tpPerMw": {
"type": "number"
},
"inputTpPerMw": {
"type": "number"
},
"outputTpPerMw": {
"type": "number"
},
"concurrency": {
"type": "number"
},
"cacheHitRate": {
"type": [
"number",
"null"
]
},
"inputTokenShare": {
"type": [
"number",
"null"
]
},
"clamped": {
"type": "boolean"
},
"clampedAbove": {
"type": "boolean"
},
"clampedBelow": {
"type": "boolean"
},
"nearest": {
"type": "object",
"properties": {
"below": {
"type": [
"object",
"null"
],
"properties": {
"interactivity": {
"type": "number"
},
"throughput": {
"type": "number"
},
"concurrency": {
"type": "number"
}
},
"additionalProperties": false
},
"above": {
"type": [
"object",
"null"
],
"properties": {
"interactivity": {
"type": "number"
},
"throughput": {
"type": "number"
},
"concurrency": {
"type": "number"
}
},
"additionalProperties": false
}
},
"required": [
"below",
"above"
],
"additionalProperties": false
},
"fleet": {
"type": [
"object",
"null"
],
"description": "Present only when mw is set.",
"properties": {
"chips": {
"type": "number"
},
"totalTokPerSec": {
"type": "number"
},
"concurrentUsers": {
"type": "number"
},
"costPerHour": {
"type": "number"
},
"costPerMonth": {
"type": "number"
}
},
"additionalProperties": false
}
},
"required": [
"hwKey",
"resultKey",
"precision",
"label",
"value",
"inputThroughput",
"outputThroughput",
"cost",
"tpPerMw",
"inputTpPerMw",
"outputTpPerMw",
"concurrency",
"cacheHitRate",
"inputTokenShare",
"clamped",
"clampedAbove",
"clampedBelow",
"nearest"
],
"additionalProperties": false
}
},
"costCap": {
"type": "array",
"description": "Present only when costcap is set.",
"items": {
"type": "object",
"properties": {
"hwKey": {
"type": "string"
},
"resultKey": {
"type": "string"
},
"label": {
"type": "string"
},
"maxInteractivity": {
"type": [
"number",
"null"
]
},
"throughput": {
"type": [
"number",
"null"
]
},
"concurrentUsers": {
"type": [
"number",
"null"
]
}
},
"required": [
"hwKey",
"resultKey",
"label",
"maxInteractivity",
"throughput",
"concurrentUsers"
],
"additionalProperties": false
}
},
"count": {
"type": "integer"
}
},
"required": [
"view",
"apiVersion",
"generatedAt",
"params",
"hardware",
"count"
],
"additionalProperties": false
}Scroll to inspect the full example
Response example
{
"view": "calculator",
"apiVersion": "v1",
"generatedAt": "2026-08-20",
"params": {
"model": "DeepSeek-V4-Pro",
"sequence": "1k/1k",
"precisions": [
"fp4"
],
"target": 35,
"mode": "interactivity-to-throughput",
"costProvider": "costh",
"costType": "total",
"percentile": "p90",
"gpus": [],
"format": "json"
},
"hardware": [
{
"hwKey": "b300_sglang",
"resultKey": "b300_sglang",
"precision": "fp4",
"label": "B300 (SGLang)",
"value": 1234.5,
"inputThroughput": 820.1,
"outputThroughput": 414.4,
"cost": {
"total": 1.21,
"input": 1.83,
"output": 3.62
},
"tpPerMw": 890123.4,
"inputTpPerMw": 591234.5,
"outputTpPerMw": 298888.9,
"concurrency": 24,
"cacheHitRate": null,
"inputTokenShare": 0.5,
"clamped": false,
"clampedAbove": false,
"clampedBelow": false,
"nearest": {
"below": {
"interactivity": 30,
"throughput": 1500,
"concurrency": 32
},
"above": {
"interactivity": 50,
"throughput": 900,
"concurrency": 16
}
}
}
],
"count": 1
}Scroll to inspect the full example
Alternate representation: text/csv
Response shape
{
"type": "string",
"description": "One flat row per hardware result."
}Response example
hwKey,resultKey,precision,label,value,inputThroughput,outputThroughput,costTotal,costInput,costOutput,tpPerMw,concurrency,clamped,clampedAbove,clampedBelow
b300_sglang,b300_sglang,fp4,B300 (SGLang),1234.5,820.1,414.4,1.21,1.83,3.62,890123.4,24,false,false,false400
Invalid parameter. The body names the parameter and, for enums, the allowed values.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"error": {
"type": "string"
},
"param": {
"type": "string"
}
},
"required": [
"error"
],
"additionalProperties": true
}Scroll to inspect the full example
Response example
{
"error": "Unknown mode: sideways",
"param": "mode",
"allowed": [
"interactivity-to-throughput",
"throughput-to-interactivity"
]
}500
Calculator view failed.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"error": {
"type": "string"
},
"param": {
"type": "string"
}
},
"required": [
"error"
],
"additionalProperties": true
}Scroll to inspect the full example
Response example
{
"error": "Internal server error"
}GETBeta/api/v1/views/fleetFleet lifecycle economics
Sizes a fixed fleet of each chip against a facility power budget at the default 35 tok/s/user target, then projects revenue, cost, and margin across the model’s life. Each chip follows its measured best-so-far staircase: every run date whose best config beat everything before it becomes a step, rolled out over the ramp. Prices default to the cheapest fleet’s break-even at a 4x output:input ratio.
Parameters
| Name | Location | Type | Requirement | Description | Example |
|---|---|---|---|---|---|
| cache | query | number | Optional | Cached-input percentage billed at the cache-read discount. Applies to agentic traces only. | 10 |
| costProvider | query | string | Optional | costh = owning, costr = renting. Profit views additionally accept custom with customCosts. No costn provider. | |
| costType | query | string | Optional | total (default), input or output tokens. Disaggregated input/output rates retain their prefill/decode denominators. | |
| format | query | enum | Optional | Response encoding. | json |
| gpus | query | CSV list | Optional | Comma-separated hardware keys; matches a full hwKey (b300_sglang) or a base chip (b300). | b300,mi355x |
| horizon | query | number | Optional | Projection horizon in months. Defaults to a short tail past the last measured sweep. | 12 |
| metric | query | enum | Optional | Which metric each point’s value field carries. | margin |
| model | query | string | Required | Display model name or compare slug. | DeepSeek-V4-Pro |
| mtbi | query | number | Optional | Mean time between interrupts, in days. | 24 |
| mw | query | number | Required | Facility power budget in MW. Must be greater than 0. | 100 |
| oprice | query | number | Optional | Output token price in $/M tokens. Defaults to 4x the input price. | 1.68 |
| percentile | query | enum | Optional | Interactivity percentile for agentic traces; fixed sequences use the median. | p90 |
| precisions | query | CSV list | Optional | Comma-separated precisions. Omit to auto-select the densest precision in the data. | fp4 |
| price | query | number | Optional | Input token price in $/M tokens. Defaults to the cheapest fleet’s break-even; giving only oprice derives it through the 4x ratio. | 0.42 |
| ramp | query | number | Optional | Months for each config rollout to reach full rate. Defaults to the dashboard lifecycle panel (0.5). | 0.5 |
| recovery | query | number | Optional | Recovery time per interrupt, in hours. | 12 |
| sequence | query | enum | Optional | Workload sequence. Accepts 1k/1k, 1k/8k, 8k/1k (or hyphenated forms) and agentic-traces (alias agentic). | 8k/1k |
| target | query | string | Optional | Positive operating-point target, tok/s/user in interactivity-to-throughput mode. Both profit views use model-specific defaults: Kimi K3 60, GLM-5.2/5.3 135, MiniMax M3 115, DeepSeek V4 Pro 170 and DeepSeek V4.1 Flash 150 tok/s/user. Dashboard links use /<model>/target/<value> route segments; this API continues to accept the target query parameter. These are editable SKU-coverage assumptions, not OpenRouter P90-equivalent measurements. | |
| tcoBasis | query | string | Optional | internal (default) or external. Applies the same configured owning/rental cost basis as the UI. |
Request
curl
curl -sS 'https://inferencex.semianalysis.com/api/v1/views/fleet?model=DeepSeek-V4-Pro&mw=100'Responses
FleetView200
One lifecycle economics series per chip: a fixed fleet sized against the power budget, projected across the model’s life with every measured config improvement.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"view": {
"type": "string",
"enum": [
"fleet"
]
},
"apiVersion": {
"type": "string",
"enum": [
"v1"
]
},
"generatedAt": {
"type": [
"string",
"null"
],
"description": "Latest run date among the history rows the view was computed from."
},
"params": {
"type": "object",
"additionalProperties": true
},
"assumptions": {
"type": "object",
"additionalProperties": true
},
"series": {
"type": "array",
"items": {
"type": "object",
"properties": {
"hwKey": {
"type": "string",
"description": "Base chip key, e.g. b300."
},
"label": {
"type": "string"
},
"hwKeysUsed": {
"type": "array",
"items": {
"type": "string"
},
"description": "Every config that contributed a rung, in takeover order."
},
"disagg": {
"type": "boolean"
},
"gpus": {
"type": "integer"
},
"provisionedMw": {
"type": "number"
},
"costPerHour": {
"type": "number"
},
"concurrentUsersNow": {
"type": "number"
},
"availability": {
"type": "number"
},
"breakEvenPricePerMTok": {
"type": [
"number",
"null"
]
},
"improvementFactor": {
"type": [
"number",
"null"
]
},
"improvementCount": {
"type": "integer"
},
"paybackMonth": {
"type": [
"number",
"null"
]
},
"lifetimeMargin": {
"type": "number"
},
"revenuePerDay": {
"type": "number"
},
"costPerDay": {
"type": "number"
},
"marginPerDay": {
"type": "number"
},
"startMonth": {
"type": "number"
},
"endMonth": {
"type": "number"
},
"rampEndMonth": {
"type": [
"number",
"null"
]
},
"points": {
"type": "array",
"items": {
"type": "object",
"properties": {
"month": {
"type": "number",
"description": "Months since the anchor date (model release)."
},
"value": {
"type": "number",
"description": "The selected metric at this point."
},
"revenue": {
"type": "number",
"description": "$/day"
},
"cost": {
"type": "number",
"description": "$/day"
},
"margin": {
"type": "number",
"description": "$/day"
},
"revenuePerMw": {
"type": "number",
"description": "$/MW/day"
},
"marginPerMw": {
"type": "number",
"description": "$/MW/day"
},
"cumulative": {
"type": "number",
"description": "Cumulative margin, $"
},
"cumulativeRevenue": {
"type": "number",
"description": "Cumulative revenue, $"
},
"isStep": {
"type": "boolean"
},
"isRamp": {
"type": "boolean"
}
},
"required": [
"month",
"value",
"revenue",
"cost",
"margin",
"revenuePerMw",
"marginPerMw",
"cumulative",
"cumulativeRevenue",
"isStep",
"isRamp"
],
"additionalProperties": false
}
}
},
"required": [
"hwKey",
"label",
"hwKeysUsed",
"disagg",
"gpus",
"provisionedMw",
"costPerHour",
"concurrentUsersNow",
"availability",
"breakEvenPricePerMTok",
"points"
],
"additionalProperties": true
}
},
"count": {
"type": "integer"
}
},
"required": [
"view",
"apiVersion",
"generatedAt",
"params",
"assumptions",
"series",
"count"
],
"additionalProperties": false
}Scroll to inspect the full example
Response example
{
"view": "fleet",
"apiVersion": "v1",
"generatedAt": "2026-08-20",
"params": {
"model": "DeepSeek-V4-Pro",
"sequence": "8k/1k",
"precisions": [
"fp4"
],
"mw": 100,
"price": 0.42,
"oprice": 1.68,
"ramp": 0.5,
"cache": 10,
"mtbi": 24,
"recovery": 12,
"horizon": 7,
"metric": "margin",
"percentile": "p90",
"gpus": [],
"format": "json"
},
"assumptions": {
"target": 35,
"costProvider": "costh",
"costType": "total",
"mtbiDays": 24,
"recoveryHours": 12,
"inputPricePerMTok": 0.42,
"outputPricePerMTok": 1.68,
"outputPriceMultiple": 4,
"rampMonths": 0.5,
"cachedInputPct": 10,
"cacheReadRatio": 1,
"availability": 0.98,
"breakEvenPricePerMTok": 0.42,
"anchorDate": "2026-04-24",
"horizonMonths": 7
},
"series": [
{
"hwKey": "b300",
"label": "B300",
"hwKeysUsed": [
"b300_sglang"
],
"disagg": false,
"gpus": 70921,
"provisionedMw": 99.99,
"costPerHour": 212763,
"concurrentUsersNow": 2500000,
"availability": 0.98,
"breakEvenPricePerMTok": 0.42,
"improvementFactor": 1.33,
"improvementCount": 1,
"paybackMonth": null,
"lifetimeMargin": 0,
"points": [
{
"month": 1.25,
"value": -1250000,
"revenue": 3860000,
"cost": 5110000,
"margin": -1250000,
"revenuePerMw": 38600,
"marginPerMw": -12500,
"cumulative": -38000000,
"cumulativeRevenue": 117000000,
"isStep": true,
"isRamp": true
}
]
}
],
"count": 1
}Scroll to inspect the full example
Alternate representation: text/csv
Response shape
{
"type": "string",
"description": "One row per lifecycle point."
}Response example
hwKey,label,month,value,revenue,cost,margin,revenuePerMw,marginPerMw,cumulative,cumulativeRevenue,isStep,isRamp
b300,B300,1.25,-1250000,3860000,5110000,-1250000,38600,-12500,-38000000,117000000,true,true400
Invalid or missing parameter. The body names the parameter and, for enums, the allowed values.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"error": {
"type": "string"
},
"param": {
"type": "string"
}
},
"required": [
"error"
],
"additionalProperties": true
}Scroll to inspect the full example
Response example
{
"error": "mw is required (facility power budget in MW, > 0)",
"param": "mw"
}500
Fleet view failed.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"error": {
"type": "string"
},
"param": {
"type": "string"
}
},
"required": [
"error"
],
"additionalProperties": true
}Scroll to inspect the full example
Response example
{
"error": "Internal server error"
}GETBeta/api/v1/views/evaluationGet the evaluation chart view
Returns the aggregated evaluation bars the /infx/evaluation dashboard renders for a model, benchmark, and run date: latest rows per config, with same-config retries averaged into one row (score is the mean; stderr covers the min/max error range; n counts the averaged runs). benchmarks lists every benchmark available for the model.
Parameters
| Name | Location | Type | Requirement | Description | Example |
|---|---|---|---|---|---|
| unofficialrun | query | string | Optional | Up to eight comma-separated public CI run IDs. Surrounding whitespace and duplicates are removed; run indices follow the remaining order. Overlay sources remain separate from official data. | |
| benchmark | query | string | Optional | Evaluation task key (e.g. gsm8k). Default: the first available benchmark for the model (alphabetical). Unknown values return 400 with the available list. | gsm8k |
| date | query | date | Optional | Requested eval run date (YYYY-MM-DD), resolved to the nearest available date like the dashboard. Default: latest available date. | 2026-08-20 |
| format | query | enum | Optional | Response encoding. csv returns one flat row per config with newline-free labels. | json |
| gpus | query | string | Optional | Comma-separated hardware/config keys; for gpu-metrics use numeric GPU indices. Omit for all. | |
| model | query | string | Required | Display model name (case-insensitive) or compare-page slug. | DeepSeek-V4-Pro |
| precisions | query | CSV list | Optional | Comma-separated precision filter (fp4, fp4fp8, fp8, bf16, int4). Default: every precision present in the model’s eval rows. | fp8 |
Request
curl
curl -sS 'https://inferencex.semianalysis.com/api/v1/views/evaluation?model=DeepSeek-V4-Pro&benchmark=gsm8k&format=json'Responses
EvaluationView200
Aggregated evaluation chart rows for the resolved model, benchmark, and date.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"view": {
"type": "string",
"enum": [
"evaluation"
]
},
"apiVersion": {
"type": "string",
"enum": [
"v1"
]
},
"params": {
"type": "object",
"properties": {
"model": {
"type": "string"
},
"benchmark": {
"type": [
"string",
"null"
]
},
"date": {
"type": [
"string",
"null"
],
"format": "date"
},
"precisions": {
"type": "array",
"items": {
"type": "string"
}
},
"gpus": {
"type": "array",
"items": {
"type": "string"
}
},
"unofficialrun": {
"type": [
"string",
"null"
]
},
"format": {
"type": "string",
"enum": [
"json",
"csv"
]
}
},
"required": [
"model",
"benchmark",
"date",
"precisions",
"gpus",
"unofficialrun",
"format"
],
"additionalProperties": false
},
"benchmarks": {
"type": "array",
"items": {
"type": "string"
}
},
"rows": {
"type": "array",
"items": {
"type": "object",
"properties": {
"source": {
"type": "string"
},
"hwKey": {
"type": "string"
},
"label": {
"type": "string"
},
"score": {
"type": "number"
},
"stderr": {
"type": "number"
},
"n": {
"type": "integer",
"description": "How many repeated runs (retries/reruns) the row averages."
},
"precision": {
"type": "string"
},
"framework": {
"type": "string"
},
"date": {
"type": "string",
"format": "date"
}
},
"required": [
"source",
"hwKey",
"label",
"score",
"stderr",
"n",
"precision",
"framework",
"date"
],
"additionalProperties": false
}
}
},
"required": [
"view",
"apiVersion",
"params",
"benchmarks",
"rows"
],
"additionalProperties": false
}Scroll to inspect the full example
Response example
{
"view": "evaluation",
"apiVersion": "v1",
"params": {
"model": "DeepSeek-V4-Pro",
"benchmark": "gsm8k",
"date": "2026-08-20",
"precisions": [
"fp8"
],
"format": "json",
"gpus": [],
"unofficialrun": null
},
"benchmarks": [
"aime25",
"gsm8k"
],
"rows": [
{
"source": "official",
"hwKey": "h200_sglang",
"label": "H200 (SGLang)\nC128 T8 E1",
"score": 0.85,
"stderr": 0.01,
"n": 2,
"precision": "fp8",
"framework": "sglang",
"date": "2026-08-20"
}
]
}Scroll to inspect the full example
Alternate representation: text/csv
Response shape
{
"type": "string"
}Response example
hwKey,label,score,stderr,n,precision,framework,date
h200_sglang,H200 (SGLang) C128 T8 E1,0.85,0.01,2,fp8,sglang,2026-08-20400
The model, benchmark, date, precisions, or format value is invalid. The body lists the allowed values.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"error": {
"type": "string"
}
},
"required": [
"error"
],
"additionalProperties": true
}Response example
{
"error": "Unknown benchmark for DeepSeek-V4-Pro: mmlu",
"param": "benchmark",
"allowed": [
"aime25",
"gsm8k"
]
}500
The evaluation query failed.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"error": {
"type": "string"
}
},
"required": [
"error"
],
"additionalProperties": true
}Response example
{
"error": "Internal server error"
}GETBeta/api/v1/views/reliabilityGet the reliability chart view
Returns the aggregated benchmark-run success rates per hardware that the /infx/reliability dashboard chart renders, bucketed by a rolling date-range preset. Success rates are percentages rounded to 2 decimal places; generatedFrom reports the first and last dates in the underlying run stats.
Parameters
| Name | Location | Type | Requirement | Description | Example |
|---|---|---|---|---|---|
| asOf | query | string | Optional | Reference date YYYY-MM-DD for reproducible rolling reliability or image-age calculations. | |
| format | query | enum | Optional | Response encoding. csv returns one flat row per hardware. | json |
| gpus | query | string | Optional | Comma-separated hardware/config keys; for gpu-metrics use numeric GPU indices. Omit for all. | |
| range | query | enum | Optional | Rolling date-range preset for the aggregation, matching the /infx/reliability dashboard presets. | last-7-days |
Request
curl
curl -sS 'https://inferencex.semianalysis.com/api/v1/views/reliability?range=last-7-days&format=json'Responses
ReliabilityView200
Aggregated success rates per hardware for the selected range.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"view": {
"type": "string",
"enum": [
"reliability"
]
},
"apiVersion": {
"type": "string",
"enum": [
"v1"
]
},
"params": {
"type": "object",
"properties": {
"range": {
"type": "string",
"enum": [
"last-3-days",
"last-7-days",
"last-month",
"last-3-months",
"all-time"
]
},
"format": {
"type": "string",
"enum": [
"json",
"csv"
]
},
"gpus": {
"type": "array",
"items": {
"type": "string"
}
},
"asOf": {
"type": [
"string",
"null"
],
"format": "date"
}
},
"required": [
"range",
"format",
"gpus",
"asOf"
],
"additionalProperties": false
},
"range": {
"type": "string",
"enum": [
"last-3-days",
"last-7-days",
"last-month",
"last-3-months",
"all-time"
]
},
"hardware": {
"type": "array",
"items": {
"type": "object",
"properties": {
"key": {
"type": "string"
},
"label": {
"type": "string"
},
"successRate": {
"type": "number",
"description": "Success percentage over the range, rounded to 2 decimal places."
},
"successes": {
"type": "integer"
},
"total": {
"type": "integer"
}
},
"required": [
"key",
"label",
"successRate",
"successes",
"total"
],
"additionalProperties": false
}
},
"generatedFrom": {
"type": "object",
"properties": {
"firstDate": {
"type": [
"string",
"null"
],
"format": "date"
},
"lastDate": {
"type": [
"string",
"null"
],
"format": "date"
}
},
"required": [
"firstDate",
"lastDate"
],
"additionalProperties": false
}
},
"required": [
"view",
"apiVersion",
"params",
"range",
"hardware",
"generatedFrom"
],
"additionalProperties": false
}Scroll to inspect the full example
Response example
{
"view": "reliability",
"apiVersion": "v1",
"params": {
"range": "last-7-days",
"format": "json",
"gpus": [],
"asOf": null
},
"range": "last-7-days",
"hardware": [
{
"key": "h200",
"label": "H200",
"successRate": 93.33,
"successes": 28,
"total": 30
},
{
"key": "b200",
"label": "B200",
"successRate": 90,
"successes": 27,
"total": 30
}
],
"generatedFrom": {
"firstDate": "2025-11-03",
"lastDate": "2026-08-26"
}
}Scroll to inspect the full example
Alternate representation: text/csv
Response shape
{
"type": "string"
}Response example
range,key,label,successRate,successes,total
last-7-days,h200,H200,93.33,28,30400
The range or format value is invalid. The body lists the allowed values.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"error": {
"type": "string"
}
},
"required": [
"error"
],
"additionalProperties": true
}Response example
{
"error": "Unknown range: last-year",
"param": "range",
"allowed": [
"last-3-days",
"last-7-days",
"last-month",
"last-3-months",
"all-time"
]
}500
The reliability query failed.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"error": {
"type": "string"
}
},
"required": [
"error"
],
"additionalProperties": true
}Response example
{
"error": "Internal server error"
}GETBeta/api/v1/views/gpu-specsGet the GPU specs view
Returns the static chip specification table behind the /infx/gpu-specs page — usable memory, bandwidth, dense tensor-core TFLOP/s, and scale-up/scale-out interconnect details — plus the chartable metric metadata. No database read; compute TFLOPS are dense (no sparsity) and memory capacities are driver-usable values. NVL72 NICs and per-GPU network bandwidth are included (GB200: ConnectX-7 400G NDR; GB300: ConnectX-8 800G XDR), while inter-rack switch/topology remain null. MI355X uses fourth-generation Infinity Fabric.
Parameters
| Name | Location | Type | Requirement | Description | Example |
|---|---|---|---|---|---|
| format | query | enum | Optional | Response encoding. csv returns one flat row per chip. | json |
| metric | query | enum | Optional | Chart metric key. When set, the response adds a ranking array of chips ordered by that metric (descending; chips without a value omitted). | memoryBandwidth |
Request
curl
curl -sS 'https://inferencex.semianalysis.com/api/v1/views/gpu-specs?metric=memoryBandwidth&format=json'Responses
GpuSpecsView200
Static chip specifications plus chart metric metadata; ranking is present only when metric is set.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"view": {
"type": "string",
"enum": [
"gpu-specs"
]
},
"apiVersion": {
"type": "string",
"enum": [
"v1"
]
},
"params": {
"type": "object",
"properties": {
"metric": {
"type": [
"string",
"null"
]
},
"format": {
"type": "string",
"enum": [
"json",
"csv"
]
}
},
"required": [
"metric",
"format"
],
"additionalProperties": false
},
"chips": {
"type": "array",
"items": {
"type": "object",
"properties": {
"key": {
"type": "string"
},
"label": {
"type": "string"
},
"name": {
"type": "string"
},
"vendor": {
"type": "string",
"enum": [
"nvidia",
"amd"
]
},
"memory": {
"type": "string"
},
"memoryType": {
"type": "string"
},
"memoryBandwidth": {
"type": "string"
},
"fp4": {
"type": [
"number",
"null"
]
},
"fp8": {
"type": "number"
},
"bf16": {
"type": "number"
},
"scaleUpTech": {
"type": "string"
},
"scaleUpBandwidth": {
"type": "string"
},
"scaleUpWorldSize": {
"type": "integer"
},
"scaleOutBandwidth": {
"type": [
"string",
"null"
]
},
"scaleOutTech": {
"type": [
"string",
"null"
]
},
"nic": {
"type": [
"string",
"null"
],
"description": "Installed NIC model and per-GPU port rate; GB200 NVL72 uses ConnectX-7 400G and GB300 NVL72 uses ConnectX-8 800G."
},
"scaleOutSwitch": {
"type": [
"string",
"null"
]
},
"scaleOutTopology": {
"type": [
"string",
"null"
],
"description": "Null means the topology is unspecified; it does not imply that a known NIC or network is absent."
},
"scaleUpTopology": {
"type": "string"
},
"scaleUpSwitch": {
"type": [
"string",
"null"
]
},
"memoryGB": {
"type": [
"number",
"null"
]
},
"memoryBandwidthTBs": {
"type": [
"number",
"null"
]
},
"fp4Tflops": {
"type": [
"number",
"null"
]
},
"fp8Tflops": {
"type": "number"
},
"bf16Tflops": {
"type": "number"
},
"scaleUpBandwidthGBs": {
"type": [
"number",
"null"
]
},
"domainMemoryTB": {
"type": "number"
},
"domainMemoryBandwidthTBs": {
"type": "number"
},
"scaleOutBandwidthGbits": {
"type": [
"number",
"null"
]
}
},
"required": [
"key",
"label",
"name",
"vendor",
"memory",
"memoryType",
"memoryBandwidth",
"fp4",
"fp8",
"bf16",
"scaleUpTech",
"scaleUpBandwidth",
"scaleUpWorldSize",
"scaleOutBandwidth",
"scaleOutTech",
"nic",
"scaleOutSwitch",
"scaleOutTopology",
"scaleUpTopology",
"scaleUpSwitch",
"memoryGB",
"memoryBandwidthTBs",
"fp4Tflops",
"fp8Tflops",
"bf16Tflops",
"scaleUpBandwidthGBs",
"domainMemoryTB",
"domainMemoryBandwidthTBs",
"scaleOutBandwidthGbits"
],
"additionalProperties": false
}
},
"metrics": {
"type": "array",
"items": {
"type": "object",
"properties": {
"key": {
"type": "string"
},
"label": {
"type": "string"
},
"unit": {
"type": "string"
}
},
"required": [
"key",
"label",
"unit"
],
"additionalProperties": false
}
},
"ranking": {
"type": "array",
"items": {
"type": "object",
"properties": {
"chip": {
"type": "string"
},
"label": {
"type": "string"
},
"value": {
"type": "number"
},
"rank": {
"type": "integer"
}
},
"required": [
"chip",
"label",
"value",
"rank"
],
"additionalProperties": false
}
}
},
"required": [
"view",
"apiVersion",
"params",
"chips",
"metrics"
],
"additionalProperties": false
}Scroll to inspect the full example
Response example
{
"view": "gpu-specs",
"apiVersion": "v1",
"params": {
"metric": "memoryBandwidth",
"format": "json"
},
"chips": [
{
"key": "b200-sxm",
"label": "B200 SXM",
"name": "B200 SXM",
"vendor": "nvidia",
"memory": "180 GB",
"memoryType": "HBM3e",
"memoryBandwidth": "8 TB/s",
"fp4": 9000,
"fp8": 4500,
"bf16": 2250,
"scaleUpTech": "NVLink 5.0",
"scaleUpBandwidth": "900 GB/s",
"scaleUpWorldSize": 8,
"scaleOutBandwidth": "400 Gbit/s",
"scaleOutTech": "gIB RoCEv2 Ethernet",
"nic": "ConnectX-7 400GbE",
"scaleOutSwitch": "12.8T Whitebox Leaf Tomahawk3 & 25.6T Whitebox Tomahawk4",
"scaleOutTopology": "4-rail optimized",
"scaleUpTopology": "Switched 2-rail Optimized",
"scaleUpSwitch": "28.8Tbit/s NVSwitch Gen 4.0",
"memoryGB": 180,
"memoryBandwidthTBs": 8,
"fp4Tflops": 9000,
"fp8Tflops": 4500,
"bf16Tflops": 2250,
"scaleUpBandwidthGBs": 900,
"domainMemoryTB": 1.44,
"domainMemoryBandwidthTBs": 64,
"scaleOutBandwidthGbits": 400
}
],
"metrics": [
{
"key": "memory",
"label": "Memory",
"unit": "GB"
},
{
"key": "memoryBandwidth",
"label": "Mem BW",
"unit": "TB/s"
}
],
"ranking": [
{
"chip": "b200-sxm",
"label": "B200 SXM",
"value": 8,
"rank": 1
}
]
}Scroll to inspect the full example
Alternate representation: text/csv
Response shape
{
"type": "string"
}Response example
key,label,name,vendor,memory,memoryType,memoryBandwidth,fp4,fp8,bf16
b200-sxm,B200 SXM,B200 SXM,nvidia,180 GB,HBM3e,8 TB/s,9000,4500,2250400
The metric or format value is invalid. The body lists the allowed values.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"error": {
"type": "string"
}
},
"required": [
"error"
],
"additionalProperties": true
}Response example
{
"error": "Unknown metric: tdp",
"param": "metric",
"allowed": [
"memory",
"memoryBandwidth",
"fp4",
"fp8",
"bf16",
"scaleUpBandwidth",
"scaleUpWorldSize",
"domainMemory",
"domainMemoryBandwidth",
"scaleOutBandwidth"
]
}Scroll to inspect the full example
500
Assembling the GPU specs payload failed.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"error": {
"type": "string"
}
},
"required": [
"error"
],
"additionalProperties": true
}Response example
{
"error": "Internal server error"
}GETBeta/api/v1/views/overviewGet the overview cost matrix view
Returns the /overview dashboard matrix as data: for every curated model-scenario row, the best community (or all-engine) serving config per hardware at the requested interactivity tier, with $/M tokens, tok/s per GPU, deltas versus a reference GPU or a historical window, and the winning config. Unlike the page, invalid parameters return 400 rather than being silently normalized.
Parameters
| Name | Location | Type | Requirement | Description | Example |
|---|---|---|---|---|---|
| compare | query | string | Optional | Comparison mode: hardware compares each cell against the reference GPU; a window (7d/30d/60d/90d) compares against the same cell that many days earlier. | 30d |
| engine | query | string | Optional | Serving-engine scope: community frameworks only, or all engines including vendor stacks. | community |
| format | query | string | Optional | Response format. CSV returns one flat row per model-scenario-hardware cell. | json |
| hwrows | query | string | Optional | Hardware-cell scope: only cells with a priced measurement, or all hardware columns. | all |
| models | query | string | Optional | Model scope: the curated default set or every model with data. | default |
| ref | query | string | Optional | Reference GPU for compare=hardware deltas. | b200 |
| rows | query | string | Optional | Row scope for historical windows: only rows that changed inside the window, or all rows. | all |
| tier | query | integer | Optional | Interactivity tier in output tokens per second per user. Cells are read at this tier. | 100 |
Request
curl
curl -sS 'https://inferencex.semianalysis.com/api/v1/views/overview?tier=50&engine=community&compare=hardware&ref=b200'Responses
OverviewView200
The overview cost matrix at the requested tier: one row per model-scenario, one cell per hardware column.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"view": {
"type": "string",
"enum": [
"overview"
]
},
"apiVersion": {
"type": "string",
"enum": [
"v1"
]
},
"generatedAt": {
"type": [
"string",
"null"
],
"format": "date"
},
"params": {
"type": "object",
"properties": {
"tier": {
"type": "integer"
},
"engine": {
"type": "string"
},
"compare": {
"type": "string"
},
"ref": {
"type": "string"
},
"models": {
"type": "string"
},
"rows": {
"type": "string"
},
"hwrows": {
"type": "string"
},
"format": {
"type": "string",
"enum": [
"json",
"csv"
]
}
},
"required": [
"tier",
"engine",
"compare",
"ref",
"models",
"rows",
"hwrows",
"format"
],
"additionalProperties": false
},
"tiers": {
"type": "array",
"items": {
"type": "integer"
}
},
"scenarios": {
"type": "array",
"items": {
"type": "string"
}
},
"referenceHardware": {
"type": "string"
},
"historicalWindow": {
"type": [
"object",
"null"
],
"properties": {
"key": {
"type": "string"
},
"snapshotDate": {
"type": "string"
},
"targetDate": {
"type": "string"
},
"earliestDate": {
"type": "string"
}
},
"additionalProperties": true
},
"unchangedRowCount": {
"type": "integer"
},
"emptyRowCount": {
"type": "integer"
},
"rows": {
"type": "array",
"items": {
"type": "object",
"properties": {
"model": {
"type": "string"
},
"modelLabel": {
"type": "string"
},
"category": {
"type": "string"
},
"scenario": {
"type": "string",
"enum": [
"single_turn_8k1k",
"agentx"
]
},
"cells": {
"type": "array",
"items": {
"type": "object",
"properties": {
"hardware": {
"type": "string"
},
"hardwareLabel": {
"type": "string"
},
"costPerMtok": {
"type": [
"number",
"null"
]
},
"throughputPerGpu": {
"type": [
"number",
"null"
]
},
"estimated": {
"type": "boolean"
},
"deltaVsRefPct": {
"type": [
"number",
"null"
]
},
"missingReason": {
"type": [
"string",
"null"
]
},
"config": {
"type": [
"object",
"null"
],
"properties": {
"framework": {
"type": "string"
},
"frameworkLabel": {
"type": "string"
},
"precision": {
"type": "string"
},
"specMethod": {
"type": "string"
},
"specLabel": {
"type": [
"string",
"null"
]
},
"disagg": {
"type": "boolean"
},
"multinode": {
"type": "boolean"
},
"latestDate": {
"type": [
"string",
"null"
]
}
},
"additionalProperties": false
},
"history": {
"type": "object",
"properties": {
"status": {
"type": "string"
},
"baselineCostPerMtok": {
"type": [
"number",
"null"
]
},
"costDeltaPct": {
"type": [
"number",
"null"
]
},
"baselineDate": {
"type": [
"string",
"null"
]
}
},
"additionalProperties": false
}
},
"required": [
"hardware",
"hardwareLabel",
"costPerMtok",
"throughputPerGpu",
"estimated",
"deltaVsRefPct",
"missingReason",
"config"
],
"additionalProperties": false
}
}
},
"required": [
"model",
"modelLabel",
"category",
"scenario",
"cells"
],
"additionalProperties": false
}
}
},
"required": [
"view",
"apiVersion",
"generatedAt",
"params",
"tiers",
"scenarios",
"referenceHardware",
"historicalWindow",
"unchangedRowCount",
"emptyRowCount",
"rows"
],
"additionalProperties": false
}Scroll to inspect the full example
Response example
{
"view": "overview",
"apiVersion": "v1",
"generatedAt": "2026-08-20",
"params": {
"tier": 50,
"engine": "community",
"compare": "hardware",
"ref": "b200",
"models": "default",
"rows": "all",
"hwrows": "all",
"format": "json"
},
"tiers": [
30,
50,
75,
100,
150,
200
],
"scenarios": [
"single_turn_8k1k",
"agentx"
],
"referenceHardware": "b200",
"historicalWindow": null,
"unchangedRowCount": 0,
"emptyRowCount": 0,
"rows": [
{
"model": "DeepSeek-V4-Pro",
"modelLabel": "DeepSeekv4 Pro 0813 1.6T",
"category": "frontier",
"scenario": "agentx",
"cells": [
{
"hardware": "b200",
"hardwareLabel": "B200",
"costPerMtok": 0.42,
"throughputPerGpu": 1315.2,
"estimated": false,
"deltaVsRefPct": null,
"missingReason": null,
"config": {
"framework": "sglang",
"frameworkLabel": "SGLang",
"precision": "fp8",
"specMethod": "mtp",
"specLabel": "MTP",
"disagg": true,
"multinode": false,
"latestDate": "2026-08-20"
}
}
]
}
]
}Scroll to inspect the full example
Alternate representation: text/csv
Response shape
{
"type": "string"
}Response example
model,scenario,tier,hardware,cost_per_mtok,throughput_per_gpu,estimated,delta_vs_ref_pct,missing_reason,framework,precision,spec_method,disagg,multinode,history_status,baseline_cost_per_mtok,history_delta_pct,baseline_date
DeepSeek-V4-Pro,agentx,50,b200,0.42,1315.2,false,,,SGLang,fp8,mtp,true,false,,,,400
A parameter value is invalid. The body names the parameter and lists the allowed values.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"error": {
"type": "string"
}
},
"required": [
"error"
],
"additionalProperties": true
}Response example
{
"error": "Unknown tier: 42",
"param": "tier",
"allowed": [
"30",
"50",
"75",
"100",
"150",
"200"
]
}500
The overview assembly failed.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"error": {
"type": "string"
}
},
"required": [
"error"
],
"additionalProperties": true
}Response example
{
"error": "Internal server error"
}GETBeta/api/v1/views/rankingsGet GPU ranking views
Returns the /rankings pages as data. Per model and scenario, hardware is ordered by its best community serving config at the entry's interactivity tier. fastest-gpu orders by tok/s per GPU and cheapest-gpu by $/M total tokens. Rows carry the winning framework, precision, and disaggregation flag, and chip links each hardware to its /chips registry slug.
Parameters
| Name | Location | Type | Requirement | Description | Example |
|---|---|---|---|---|---|
| format | query | string | Optional | Response format. CSV returns one flat row per ranked hardware. | json |
| kind | query | string | Optional | Ranking kind: fastest-gpu orders by tok/s per GPU (descending), cheapest-gpu by $/M total tokens (ascending). | cheapest-gpu |
| model | query | string | Optional | Display model name (case-insensitive) or compare-page slug. Default: every ranked model; models without measurable rows are dropped there but kept when requested explicitly. | DeepSeek-V4-Pro |
| scenario | query | string | Optional | Workload scenario. Aliases: 8k-1k for single_turn_8k1k, agentic for agentx. Default: every curated overview scenario per model (the rows the /overview matrix shows). | agentx |
Request
curl
curl -sS 'https://inferencex.semianalysis.com/api/v1/views/rankings?kind=cheapest-gpu&model=DeepSeek-V4-Pro&scenario=agentx'Responses
RankingsView200
Ranked hardware per model-scenario, community engine scope. Each entry reports the interactivity tier (tok/s/user) it is ranked at. The default is 125. One of 50, 75, 100, 150, 175 and 200 replaces it only when that tier measures more chips than 125, taking the tier with the most chips and the lowest on a tie. When no tier measures at least two chips, the entry has no rows.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"view": {
"type": "string",
"enum": [
"rankings"
]
},
"apiVersion": {
"type": "string",
"enum": [
"v1"
]
},
"generatedAt": {
"type": [
"string",
"null"
],
"format": "date"
},
"params": {
"type": "object",
"properties": {
"kind": {
"type": "string"
},
"model": {
"type": "string"
},
"scenario": {
"type": "string"
},
"engine": {
"type": "string"
},
"format": {
"type": "string",
"enum": [
"json",
"csv"
]
}
},
"required": [
"kind",
"model",
"scenario",
"engine",
"format"
],
"additionalProperties": false
},
"kind": {
"type": "string",
"enum": [
"fastest-gpu",
"cheapest-gpu"
]
},
"entries": {
"type": "array",
"items": {
"type": "object",
"properties": {
"model": {
"type": "string"
},
"modelSlug": {
"type": "string"
},
"modelLabel": {
"type": "string"
},
"scenario": {
"type": "string",
"enum": [
"single_turn_8k1k",
"agentx"
]
},
"tier": {
"type": "integer"
},
"rows": {
"type": "array",
"items": {
"type": "object",
"properties": {
"rank": {
"type": "integer"
},
"hardware": {
"type": "string"
},
"hardwareLabel": {
"type": "string"
},
"chip": {
"type": [
"string",
"null"
]
},
"value": {
"type": [
"number",
"null"
]
},
"unit": {
"type": "string",
"enum": [
"tokens_per_second_per_gpu",
"usd_per_million_tokens"
]
},
"framework": {
"type": [
"string",
"null"
]
},
"precision": {
"type": [
"string",
"null"
]
},
"disagg": {
"type": [
"boolean",
"null"
]
}
},
"required": [
"rank",
"hardware",
"hardwareLabel",
"chip",
"value",
"unit",
"framework",
"precision",
"disagg"
],
"additionalProperties": false
}
}
},
"required": [
"model",
"modelSlug",
"modelLabel",
"scenario",
"tier",
"rows"
],
"additionalProperties": false
}
}
},
"required": [
"view",
"apiVersion",
"generatedAt",
"params",
"kind",
"entries"
],
"additionalProperties": false
}Scroll to inspect the full example
Response example
{
"view": "rankings",
"apiVersion": "v1",
"generatedAt": "2026-08-20",
"params": {
"kind": "cheapest-gpu",
"model": "DeepSeek-V4-Pro",
"scenario": "agentx",
"engine": "community",
"format": "json"
},
"kind": "cheapest-gpu",
"entries": [
{
"model": "DeepSeek-V4-Pro",
"modelSlug": "deepseek-v4",
"modelLabel": "DeepSeekv4 Pro 0813 1.6T",
"scenario": "agentx",
"tier": 50,
"rows": [
{
"rank": 1,
"hardware": "b200",
"hardwareLabel": "B200",
"chip": "b200",
"value": 0.42,
"unit": "usd_per_million_tokens",
"framework": "SGLang",
"precision": "fp8",
"disagg": true
}
]
}
]
}Scroll to inspect the full example
Alternate representation: text/csv
Response shape
{
"type": "string"
}Response example
kind,model,model_slug,scenario,tier,rank,hardware,hardware_label,chip,value,unit,framework,precision,disagg
cheapest-gpu,DeepSeek-V4-Pro,deepseek-v4,agentx,50,1,b200,B200,b200,0.42,usd_per_million_tokens,SGLang,fp8,true400
The kind, model, scenario, or format value is invalid. The body names the parameter and lists the allowed values.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"error": {
"type": "string"
}
},
"required": [
"error"
],
"additionalProperties": true
}Response example
{
"error": "Unknown kind: slowest-gpu",
"param": "kind",
"allowed": [
"fastest-gpu",
"cheapest-gpu"
]
}500
The rankings query failed.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"error": {
"type": "string"
}
},
"required": [
"error"
],
"additionalProperties": true
}Response example
{
"error": "Internal server error"
}GETBeta/api/v1/views/compareGet a GPU pair comparison view
Returns a /compare page as data: for one model and two GPUs, interpolated throughput, cost, and efficiency at each interactivity tier from the same pipeline the page renders, with a per-tier winner and an overall head-to-head summary. Supports custom tiers and per-dollar, precision, and spec-decode variants.
Parameters
| Name | Location | Type | Requirement | Description | Example |
|---|---|---|---|---|---|
| format | query | string | Optional | Response format. CSV returns one flat row per tier. | json |
| gpus | query | CSV GPU list | Optional | Exactly 2 distinct GPU base keys, comma-separated (e.g. b200,mi355x). Unknown keys return 400 with the allowed list. | b200,mi355x |
| model | query | string | Optional | Display model name (case-insensitive) or compare-page model slug. Use together with gpus as an alternative to slug. | DeepSeek-V4-Pro |
| scenario | query | string | Optional | Workload sequence. Aliases: 8k-1k, 1k-1k, 1k-8k, agentic. Default: the pair default the page picks (AgentX-featured models fall back to agentic-traces, others to 8k/1k). | 8k/1k |
| slug | query | string | Optional | Compare-page slug, <model>-<gpuA>-vs-<gpuB>. Either slug or model+gpus is required; GPU order is canonicalized like the page redirect. | deepseek-v4-b200-vs-mi355x |
| tiers | query | CSV number list | Optional | Custom interactivity targets (tok/s/user), comma-separated, max 12. Default: the 3 page targets spanning the shared measured range. Requested tiers outside the measured range are dropped. | 25,50,75 |
| variant | query | string | Optional | View variant. per-dollar switches the winner/delta basis from throughput to $/M tokens; precision adds a per-precision head-to-head summary; spec-decode adds a per-speculative-decoding-method breakdown (fixed sequences only). | per-dollar |
Request
curl
curl -sS 'https://inferencex.semianalysis.com/api/v1/views/compare?slug=deepseek-v4-b200-vs-mi355x'Responses
CompareView200
Head-to-head interpolated table for the GPU pair at the effective scenario and precision, plus per-side summaries.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"view": {
"type": "string",
"enum": [
"compare"
]
},
"apiVersion": {
"type": "string",
"enum": [
"v1"
]
},
"generatedAt": {
"type": [
"string",
"null"
],
"format": "date"
},
"params": {
"type": "object",
"properties": {
"slug": {
"type": "string"
},
"model": {
"type": "string"
},
"gpus": {
"type": "array",
"items": {
"type": "string"
}
},
"scenario": {
"type": "string"
},
"variant": {
"type": "string"
},
"tiers": {
"oneOf": [
{
"type": "string",
"enum": [
"default"
]
},
{
"type": "array",
"items": {
"type": "number"
}
}
]
},
"format": {
"type": "string",
"enum": [
"json",
"csv"
]
}
},
"additionalProperties": false
},
"model": {
"type": "object",
"properties": {
"slug": {
"type": "string"
},
"displayName": {
"type": "string"
},
"label": {
"type": "string"
}
},
"required": [
"slug",
"displayName",
"label"
],
"additionalProperties": false
},
"gpus": {
"type": "array",
"items": {
"type": "string"
}
},
"scenario": {
"type": [
"string",
"null"
]
},
"precision": {
"type": [
"string",
"null"
]
},
"variant": {
"type": "string"
},
"tiers": {
"type": "array",
"items": {
"type": "number"
}
},
"interactivityRange": {
"type": "object",
"properties": {
"min": {
"type": "number"
},
"max": {
"type": "number"
}
},
"required": [
"min",
"max"
],
"additionalProperties": false
},
"dataRange": {
"type": "object",
"properties": {
"oldest": {
"type": [
"string",
"null"
]
},
"newest": {
"type": [
"string",
"null"
]
}
},
"required": [
"oldest",
"newest"
],
"additionalProperties": false
},
"table": {
"type": "array",
"items": {
"type": "object",
"properties": {
"tier": {
"type": "number"
},
"a": {
"type": [
"object",
"null"
],
"properties": {
"hardware": {
"type": "string"
},
"configKey": {
"type": "string"
},
"throughputPerGpu": {
"type": "number"
},
"inputThroughputPerGpu": {
"type": "number"
},
"outputThroughputPerGpu": {
"type": "number"
},
"costPerMtok": {
"type": "number"
},
"costPerMtokInput": {
"type": "number"
},
"costPerMtokOutput": {
"type": "number"
},
"throughputPerMw": {
"type": "number"
},
"concurrency": {
"type": "number"
},
"precision": {
"type": [
"string",
"null"
]
},
"clamped": {
"type": "boolean"
}
},
"additionalProperties": false
},
"b": {
"type": [
"object",
"null"
],
"properties": {
"hardware": {
"type": "string"
},
"configKey": {
"type": "string"
},
"throughputPerGpu": {
"type": "number"
},
"inputThroughputPerGpu": {
"type": "number"
},
"outputThroughputPerGpu": {
"type": "number"
},
"costPerMtok": {
"type": "number"
},
"costPerMtokInput": {
"type": "number"
},
"costPerMtokOutput": {
"type": "number"
},
"throughputPerMw": {
"type": "number"
},
"concurrency": {
"type": "number"
},
"precision": {
"type": [
"string",
"null"
]
},
"clamped": {
"type": "boolean"
}
},
"additionalProperties": false
},
"basis": {
"type": "string",
"enum": [
"throughputPerGpu",
"costPerMtok"
]
},
"deltaPct": {
"type": [
"number",
"null"
]
},
"winner": {
"type": [
"string",
"null"
]
}
},
"required": [
"tier",
"a",
"b",
"basis",
"deltaPct",
"winner"
],
"additionalProperties": false
}
},
"summary": {
"type": "object",
"properties": {
"a": {
"type": "object",
"properties": {
"hardware": {
"type": "string"
},
"configCount": {
"type": "integer"
},
"bestThroughputPerGpu": {
"type": [
"number",
"null"
]
},
"bestMedianTtft": {
"type": [
"number",
"null"
]
},
"bestMedianTpot": {
"type": [
"number",
"null"
]
}
},
"additionalProperties": true
},
"b": {
"type": "object",
"properties": {
"hardware": {
"type": "string"
},
"configCount": {
"type": "integer"
},
"bestThroughputPerGpu": {
"type": [
"number",
"null"
]
},
"bestMedianTtft": {
"type": [
"number",
"null"
]
},
"bestMedianTpot": {
"type": [
"number",
"null"
]
}
},
"additionalProperties": true
},
"headToHead": {
"type": [
"object",
"null"
],
"properties": {
"faster": {
"type": "string"
},
"slower": {
"type": "string"
},
"tputPct": {
"type": "number"
},
"cheaper": {
"type": "string"
},
"pricier": {
"type": "string"
},
"costPct": {
"type": "number"
}
},
"additionalProperties": false
},
"byPrecision": {
"type": "array",
"items": {
"type": "object",
"properties": {
"precision": {
"type": "string"
},
"tiers": {
"type": "array",
"items": {
"type": "number"
}
},
"headToHead": {
"type": [
"object",
"null"
],
"properties": {
"faster": {
"type": "string"
},
"slower": {
"type": "string"
},
"tputPct": {
"type": "number"
},
"cheaper": {
"type": "string"
},
"pricier": {
"type": "string"
},
"costPct": {
"type": "number"
}
},
"additionalProperties": false
}
},
"required": [
"precision",
"tiers",
"headToHead"
],
"additionalProperties": false
}
},
"bySpecDecode": {
"type": "array",
"items": {
"type": "object",
"properties": {
"specMethod": {
"type": "string"
},
"tier": {
"type": "number"
},
"a": {
"type": [
"object",
"null"
],
"properties": {
"hardware": {
"type": "string"
},
"configKey": {
"type": "string"
},
"throughputPerGpu": {
"type": "number"
},
"inputThroughputPerGpu": {
"type": "number"
},
"outputThroughputPerGpu": {
"type": "number"
},
"costPerMtok": {
"type": "number"
},
"costPerMtokInput": {
"type": "number"
},
"costPerMtokOutput": {
"type": "number"
},
"throughputPerMw": {
"type": "number"
},
"concurrency": {
"type": "number"
},
"precision": {
"type": [
"string",
"null"
]
},
"clamped": {
"type": "boolean"
}
},
"additionalProperties": false
},
"b": {
"type": [
"object",
"null"
],
"properties": {
"hardware": {
"type": "string"
},
"configKey": {
"type": "string"
},
"throughputPerGpu": {
"type": "number"
},
"inputThroughputPerGpu": {
"type": "number"
},
"outputThroughputPerGpu": {
"type": "number"
},
"costPerMtok": {
"type": "number"
},
"costPerMtokInput": {
"type": "number"
},
"costPerMtokOutput": {
"type": "number"
},
"throughputPerMw": {
"type": "number"
},
"concurrency": {
"type": "number"
},
"precision": {
"type": [
"string",
"null"
]
},
"clamped": {
"type": "boolean"
}
},
"additionalProperties": false
}
},
"required": [
"specMethod",
"tier",
"a",
"b"
],
"additionalProperties": false
}
}
},
"required": [
"a",
"b",
"headToHead"
],
"additionalProperties": false
}
},
"required": [
"view",
"apiVersion",
"generatedAt",
"params",
"model",
"gpus",
"scenario",
"precision",
"variant",
"tiers",
"interactivityRange",
"dataRange",
"table",
"summary"
],
"additionalProperties": false
}Scroll to inspect the full example
Response example
{
"view": "compare",
"apiVersion": "v1",
"generatedAt": "2026-08-20",
"params": {
"slug": "deepseek-v4-b200-vs-mi355x",
"model": "DeepSeek-V4-Pro",
"gpus": [
"b200",
"mi355x"
],
"scenario": "auto",
"variant": "default",
"tiers": "default",
"format": "json"
},
"model": {
"slug": "deepseek-v4",
"displayName": "DeepSeek-V4-Pro",
"label": "DeepSeekv4 Pro 0813 1.6T"
},
"gpus": [
"b200",
"mi355x"
],
"scenario": "agentic-traces",
"precision": "fp8",
"variant": "default",
"tiers": [
21,
34,
47
],
"interactivityRange": {
"min": 12,
"max": 55
},
"dataRange": {
"oldest": "2026-05-02",
"newest": "2026-08-20"
},
"table": [
{
"tier": 34,
"a": {
"hardware": "b200",
"configKey": "b200_sglang",
"throughputPerGpu": 1315.2,
"inputThroughputPerGpu": 9821.4,
"outputThroughputPerGpu": 1315.2,
"costPerMtok": 0.42,
"costPerMtokInput": 0.06,
"costPerMtokOutput": 0.48,
"throughputPerMw": 985000,
"concurrency": 96,
"precision": "fp8",
"clamped": false
},
"b": {
"hardware": "mi355x",
"configKey": "mi355x_sglang",
"throughputPerGpu": 1104.7,
"inputThroughputPerGpu": 8455.1,
"outputThroughputPerGpu": 1104.7,
"costPerMtok": 0.47,
"costPerMtokInput": 0.07,
"costPerMtokOutput": 0.53,
"throughputPerMw": 912000,
"concurrency": 88,
"precision": "fp8",
"clamped": false
},
"basis": "throughputPerGpu",
"deltaPct": 19.1,
"winner": "b200"
}
],
"summary": {
"a": {
"hardware": "b200",
"configCount": 12,
"bestThroughputPerGpu": 1840.3,
"bestMedianTtft": 0.42,
"bestMedianTpot": 8.1
},
"b": {
"hardware": "mi355x",
"configCount": 9,
"bestThroughputPerGpu": 1512.8,
"bestMedianTtft": 0.51,
"bestMedianTpot": 9.4
},
"headToHead": {
"faster": "B200",
"slower": "MI355X",
"tputPct": 19,
"cheaper": "B200",
"pricier": "MI355X",
"costPct": 11
}
}
}Scroll to inspect the full example
Alternate representation: text/csv
Response shape
{
"type": "string"
}Response example
model,scenario,tier,basis,delta_pct,winner,a_hardware,a_throughput_per_gpu,a_cost_per_mtok,a_concurrency,a_clamped,b_hardware,b_throughput_per_gpu,b_cost_per_mtok,b_concurrency,b_clamped
DeepSeek-V4-Pro,agentic-traces,34,throughputPerGpu,19.1,b200,b200,1315.2,0.42,96,false,mi355x,1104.7,0.47,88,false400
The slug, model, gpus, scenario, variant, tiers, or format value is invalid. The body names the parameter and lists the allowed values where applicable.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"error": {
"type": "string"
}
},
"required": [
"error"
],
"additionalProperties": true
}Response example
{
"error": "Unknown compare slug: not-a-pair. Expected <model>-<gpuA>-vs-<gpuB>, e.g. deepseek-v4-b200-vs-mi355x.",
"param": "slug"
}500
The comparison assembly failed.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"error": {
"type": "string"
}
},
"required": [
"error"
],
"additionalProperties": true
}Response example
{
"error": "Internal server error"
}GETBeta/api/v1/views/operatorxGet the OperatorX view
OperatorX kernel-level measurements behind the /opx page: GEMM, MoE and attention ops timed on each GPU, aligned by case across hardware. One row per case and GPU, from that GPU’s newest stored run. Cases come from one workload source or from every case one model contributes, at one device split (parallel). flops and bytes are useful work per GPU (each data-parallel group’s work divided by tp), so tflops and tb_s compare across splits; they are null where the op has no defined count. roofline_us is the least time the case can take on that GPU: each stage (projection, attention core, experts, router, ...) bound by compute at the peak for its precision or by memory bandwidth, plus its collectives over the scale-up link; roofline_share is roofline_us over latency_us. run_id and result_index identify the stored result. Cached until new results are ingested; a response missing an unreadable run is not cached.
Parameters
| Name | Location | Type | Requirement | Description | Example |
|---|---|---|---|---|---|
| format | query | enum | Optional | Response encoding. csv returns one flat row per case and GPU; sources and dims are joined into text. | json |
| hardware | query | string | Optional | Comma-separated GPU keys from options.hardware (h200, b200, mi355x, ...). Omit for all. | b200,h200 |
| model | query | string | Optional | Model name from options.models, case-insensitive: every case the model contributes, across workloads. Mutually exclusive with workload. | DeepSeek-V4-Pro |
| op | query | enum | Optional | Op family: gemm (default), moe, or attention (every attention module type). | attention |
| page | query | string | Optional | Zero-based JSON page of 100 rows. CSV returns every matching row. | 0 |
| parallel | query | string | Optional | Device split key from options.parallel: 1 for one device (tp=dp=ep=dcp=1), else axes above 1 such as tp8 or dp8·ep8. Defaults to one device, else the smallest split present. | 1 |
| status | query | enum | Optional | Result status: ok (default), unsupported, error, missing, or all. A GPU that never ran a case has no row for it. | ok |
| workload | query | string | Optional | Workload source id from options.workloads, such as attention:Kimi-K3. Defaults to the best-covered workload. Mutually exclusive with model. | attention:DeepSeek-V4-Pro |
Request
curl
curl -sS 'https://inferencex.semianalysis.com/api/v1/views/operatorx?op=attention&model=DeepSeek-V4-Pro'Responses
OperatorXView200
The resolved selection, its options, and one row per case and GPU with that GPU’s newest stored result.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"view": {
"type": "string",
"enum": [
"operatorx"
]
},
"apiVersion": {
"type": "string",
"enum": [
"v1"
]
},
"params": {
"type": "object",
"properties": {
"op": {
"type": "string"
},
"workload": {
"type": [
"string",
"null"
]
},
"model": {
"type": [
"string",
"null"
]
},
"parallel": {
"type": "string"
},
"hardware": {
"type": "array",
"items": {
"type": "string"
}
},
"status": {
"type": "string"
},
"page": {
"type": "integer"
},
"format": {
"type": "string",
"enum": [
"json",
"csv"
]
}
},
"required": [
"op",
"workload",
"model",
"parallel",
"hardware",
"status",
"page",
"format"
],
"additionalProperties": false
},
"options": {
"type": "object",
"properties": {
"ops": {
"type": "array",
"items": {
"type": "string"
}
},
"workloads": {
"type": "array",
"items": {
"type": "object",
"properties": {
"id": {
"type": "string"
},
"label": {
"type": "string"
},
"cases": {
"type": "integer"
},
"hardware": {
"type": "array",
"items": {
"type": "string"
}
}
},
"required": [
"id",
"label",
"cases",
"hardware"
],
"additionalProperties": false
}
},
"models": {
"type": "array",
"items": {
"type": "string"
}
},
"parallel": {
"type": "array",
"items": {
"type": "object",
"properties": {
"key": {
"type": "string"
},
"label": {
"type": "string"
},
"cases": {
"type": "integer"
}
},
"required": [
"key",
"label",
"cases"
],
"additionalProperties": false
}
},
"hardware": {
"type": "array",
"items": {
"type": "object",
"properties": {
"id": {
"type": "string"
},
"runner": {
"type": "string"
},
"runs": {
"type": "array",
"items": {
"type": "object",
"properties": {
"runId": {
"type": "string"
},
"generatedAt": {
"type": "string"
}
},
"required": [
"runId",
"generatedAt"
],
"additionalProperties": false
}
}
},
"required": [
"id",
"runner",
"runs"
],
"additionalProperties": false
}
}
},
"required": [
"ops",
"workloads",
"models",
"parallel",
"hardware"
],
"additionalProperties": false
},
"url": {
"type": "string"
},
"total": {
"type": "integer"
},
"pageSize": {
"type": "integer"
},
"rows": {
"type": "array",
"items": {
"type": "object",
"properties": {
"case_key": {
"type": "string"
},
"op_type": {
"type": "string",
"enum": [
"gemm",
"moe",
"mla",
"mla_dsa",
"dsv4_attn",
"gqa",
"qsa",
"gdn",
"kda"
]
},
"parallel": {
"type": "string"
},
"testlist": {
"type": "string"
},
"sources": {
"type": "array",
"items": {
"type": "object",
"properties": {
"model": {
"type": "string"
},
"roles": {
"type": "array",
"items": {
"type": "string"
}
}
},
"required": [
"model",
"roles"
],
"additionalProperties": false
}
},
"shape": {
"type": "string"
},
"precision": {
"type": "string"
},
"compute_precision": {
"type": "string",
"enum": [
"fp4",
"fp8",
"bf16",
"other"
]
},
"x": {
"type": [
"number",
"null"
]
},
"dims": {
"type": "object",
"additionalProperties": {
"type": "number"
}
},
"flops": {
"type": [
"number",
"null"
]
},
"bytes": {
"type": [
"number",
"null"
]
},
"hardware": {
"type": "string"
},
"status": {
"type": "string",
"enum": [
"ok",
"unsupported",
"error",
"missing"
]
},
"latency_us": {
"type": [
"number",
"null"
]
},
"tflops": {
"type": [
"number",
"null"
]
},
"tb_s": {
"type": [
"number",
"null"
]
},
"roofline_us": {
"type": [
"number",
"null"
]
},
"roofline_share": {
"type": [
"number",
"null"
]
},
"kernel": {
"type": [
"string",
"null"
]
},
"cuda_graph": {
"type": [
"boolean",
"null"
]
},
"run_id": {
"type": [
"string",
"null"
]
},
"result_index": {
"type": [
"integer",
"null"
]
},
"revision": {
"type": [
"string",
"null"
]
}
},
"required": [
"case_key",
"op_type",
"parallel",
"testlist",
"sources",
"shape",
"precision",
"compute_precision",
"x",
"dims",
"flops",
"bytes",
"hardware",
"status",
"latency_us",
"tflops",
"tb_s",
"roofline_us",
"roofline_share",
"kernel",
"cuda_graph",
"run_id",
"result_index",
"revision"
],
"additionalProperties": false
}
}
},
"required": [
"view",
"apiVersion",
"params",
"options",
"url",
"total",
"pageSize",
"rows"
],
"additionalProperties": false
}Scroll to inspect the full example
Response example
{
"view": "operatorx",
"apiVersion": "v1",
"params": {
"op": "attention",
"workload": null,
"model": "DeepSeek-V4-Pro",
"parallel": "1",
"hardware": [
"b200",
"h200"
],
"status": "ok",
"page": 0,
"format": "json"
},
"options": {
"ops": [
"gemm",
"moe",
"attention"
],
"workloads": [
{
"id": "attention:DeepSeek-V4-Pro",
"label": "DeepSeek-V4-Pro attention",
"cases": 99,
"hardware": [
"b200",
"h200"
]
}
],
"models": [
"DeepSeek-V4-Pro"
],
"parallel": [
{
"key": "1",
"label": "1 GPU",
"cases": 99
}
],
"hardware": [
{
"id": "b200",
"runner": "b200-dgxc",
"runs": [
{
"runId": "36372535030",
"generatedAt": "2026-09-27T12:00:00Z"
}
]
}
]
},
"url": "/opx/model?model=DeepSeek-V4-Pro",
"total": 1,
"pageSize": 100,
"rows": [
{
"case_key": "{\"batch\":{\"groups\":[{\"count\":1,\"ctx\":1024,\"q\":1}]},...}|vllm",
"op_type": "dsv4_attn",
"parallel": "1",
"testlist": "attn_deepseek_v4",
"sources": [
{
"model": "DeepSeek-V4-Pro",
"roles": [
"attn"
]
}
],
"shape": "V4 attention ÷4 · 1×q1 ctx1024",
"precision": "proj e4m3 (128×128 ue8m0) · kv fp8",
"compute_precision": "fp8",
"x": 1,
"dims": {
"tokens": 1,
"hidden": 7168,
"heads": 128,
"compressRatio": 4
},
"flops": 1200000000,
"bytes": 110000000,
"hardware": "b200",
"status": "ok",
"latency_us": 41.2,
"tflops": 29.1,
"tb_s": 2.67,
"roofline_us": 18.3,
"roofline_share": 0.44,
"kernel": "flash_mla_sparse_fwd",
"cuda_graph": true,
"run_id": "36372535030",
"result_index": 66,
"revision": "63"
}
]
}Scroll to inspect the full example
Alternate representation: text/csv
Response shape
{
"type": "string"
}Response example
case_key,op_type,parallel,testlist,sources,shape,precision,compute_precision,x,dims,flops,bytes,hardware,status,latency_us,tflops,tb_s
...,dsv4_attn,1,attn_deepseek_v4,DeepSeek-V4-Pro: attn,V4 attention ÷4 · 1×q1 ctx1024,proj e4m3 (128×128 ue8m0) · kv fp8,fp8,1,tokens=1 hidden=7168 heads=128 compressRatio=4,1200000000,110000000,b200,ok,41.2,29.1,2.67400
A parameter is unknown, repeated, or out of range. The body lists the allowed values where they are known.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"error": {
"type": "string"
}
},
"required": [
"error"
],
"additionalProperties": true
}Response example
{
"error": "Unknown parallel: tp4",
"param": "parallel",
"allowed": [
"1",
"tp8"
]
}503
The OperatorX database could not be read. Not cached.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"error": {
"type": "string"
}
},
"required": [
"error"
],
"additionalProperties": true
}Response example
{
"error": "Source data unavailable"
}500
Assembling the view failed.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"error": {
"type": "string"
}
},
"required": [
"error"
],
"additionalProperties": true
}Response example
{
"error": "Internal server error"
}GETBeta/api/v1/views/first-tokenFirst-token winners
Read-only first-token winners using the dashboard's source handlers and calculation helpers. Unknown and repeated query keys return 400. The response includes resolved params and preserves missing evidence. Renderer-only styling is not an API parameter.
Parameters
| Name | Location | Type | Requirement | Description | Example |
|---|---|---|---|---|---|
| chipBasis | query | string | Optional | pool (default) or total. Chips that divide disaggregated input and output throughput and every metric derived from them: pool uses prefill chips for input and decode chips for output; total uses every chip in the deployment, the basis aggregated configs and tput_per_gpu always use. Aggregated rows are identical under both. | |
| caps | query | string | Optional | One to eight positive finite first-token caps in seconds, sorted and deduplicated; default 2,5,10,15,20. Any invalid entry returns 400. | |
| costProvider | query | string | Optional | costh = owning, costr = renting. Profit views additionally accept custom with customCosts. No costn provider. | |
| costType | query | string | Optional | total (default), input or output tokens. Disaggregated input/output rates retain their prefill/decode denominators. | |
| date | query | string | Optional | Snapshot cutoff date, YYYY-MM-DD. runId selects an exact logical run snapshot instead. | |
| gpus | query | string | Optional | Comma-separated hardware/config keys; for gpu-metrics use numeric GPU indices. Omit for all. | |
| minInteractivity | query | string | Optional | Minimum tok/s/user, default 150 for AgentX or 35 for fixed-length workloads. | |
| model | query | string | Required | Model display name or comparison slug, case-insensitive. Required for benchmark-based views. The image view also accepts current image-catalog names, trims whitespace and defaults to all; unknown models return 400. | DeepSeek-V4-Pro |
| percentile | query | string | Optional | AgentX latency percentile p75 or p90; CollectiveX supports p50, p95, p99. | |
| precisions | query | string | Optional | Comma-separated precision keys; omitted selection uses available curve density. Calculator extensions auto-select the densest official precision and include precisions present in unofficial-run overlays. | |
| runId | query | string | Optional | Positive safe integer workflow run ID. Omit to select the default run; required for GPU metrics. | 123 |
| sequence | query | string | Optional | Workload: 1k/1k, 1k/8k, 8k/1k or agentic-traces. AgentX extension views default to agentic-traces. | |
| tcoBasis | query | string | Optional | internal (default) or external. Applies the same configured owning/rental cost basis as the UI. | |
| unofficialrun | query | string | Optional | Up to eight comma-separated public CI run IDs. Surrounding whitespace and duplicates are removed; run indices follow the remaining order. Overlay sources remain separate from official data. |
Request
curl
curl -sS 'https://inferencex.semianalysis.com/api/v1/views/first-token?model=DeepSeek-V4-Pro'Responses
FirstTokenView200
Resolved selection and public view data.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"view": {
"type": "string",
"enum": [
"first-token"
]
},
"apiVersion": {
"type": "string",
"enum": [
"v1"
]
},
"params": {
"type": "object",
"additionalProperties": true
},
"data": {
"type": "object",
"additionalProperties": true
}
},
"required": [
"view",
"apiVersion",
"params"
],
"additionalProperties": true
}Scroll to inspect the full example
Response example
{
"view": "first-token",
"apiVersion": "v1",
"params": {}
}404
Source status is preserved; internal details are removed and failures are not cached.
Media type: application/jsonResponse shape
{
"type": "object",
"additionalProperties": true
}Response example
{
"error": "Source data unavailable"
}429
Source status is preserved; internal details are removed and failures are not cached.
Media type: application/jsonResponse shape
{
"type": "object",
"additionalProperties": true
}Response example
{
"error": "Source data unavailable"
}503
Source status is preserved; internal details are removed and failures are not cached.
Media type: application/jsonResponse shape
{
"type": "object",
"additionalProperties": true
}Response example
{
"error": "Source data unavailable"
}400
Invalid or unsupported query.
Media type: application/jsonResponse shape
{
"type": "object",
"additionalProperties": true
}Response example
{
"error": "Invalid parameter",
"param": "model"
}500
Data retrieval or projection failed.
Media type: application/jsonResponse shape
{
"type": "object",
"additionalProperties": true
}Response example
{
"error": "Internal server error"
}GETBeta/api/v1/views/cache-reuseCache-reuse curves
Read-only cache-reuse curves using the dashboard's source handlers and calculation helpers. Unknown and repeated query keys return 400. The response includes resolved params and preserves missing evidence. Renderer-only styling is not an API parameter.
Parameters
| Name | Location | Type | Requirement | Description | Example |
|---|---|---|---|---|---|
| config | query | string | Optional | Exact cache-reuse configuration key from configurations. Omit for the shared dashboard default. | |
| date | query | string | Optional | Snapshot cutoff date, YYYY-MM-DD. runId selects an exact logical run snapshot instead. | |
| gpus | query | string | Optional | Comma-separated hardware/config keys; for gpu-metrics use numeric GPU indices. Omit for all. | |
| model | query | string | Required | Model display name or comparison slug, case-insensitive. Required for benchmark-based views. The image view also accepts current image-catalog names, trims whitespace and defaults to all; unknown models return 400. | DeepSeek-V4-Pro |
| percentile | query | string | Optional | AgentX latency percentile p75 or p90; CollectiveX supports p50, p95, p99. | |
| precisions | query | string | Optional | Comma-separated precision keys; omitted selection uses available curve density. Calculator extensions auto-select the densest official precision and include precisions present in unofficial-run overlays. | |
| recipe | query | string | Optional | Exact key from data.recipes (dashboard c_recipe). Missing or unknown keys select the recipe with the most tiered rows, then most rows, then label; overlays supply choices only without official rows. params.recipe reports the resolved key. | |
| runId | query | string | Optional | Positive safe integer workflow run ID. Omit to select the default run; required for GPU metrics. | 123 |
| sequence | query | string | Optional | Workload: 1k/1k, 1k/8k, 8k/1k or agentic-traces. AgentX extension views default to agentic-traces. | |
| tcoBasis | query | string | Optional | internal (default) or external. Applies the same configured owning/rental cost basis as the UI. | |
| unofficialrun | query | string | Optional | Up to eight comma-separated public CI run IDs. Surrounding whitespace and duplicates are removed; run indices follow the remaining order. Overlay sources remain separate from official data. |
Request
curl
curl -sS 'https://inferencex.semianalysis.com/api/v1/views/cache-reuse?model=DeepSeek-V4-Pro'Responses
CacheReuseView200
Resolved selection and public view data.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"view": {
"type": "string",
"enum": [
"cache-reuse"
]
},
"apiVersion": {
"type": "string",
"enum": [
"v1"
]
},
"params": {
"type": "object",
"additionalProperties": true
},
"configurations": {
"type": "object",
"additionalProperties": true
},
"data": {
"type": [
"object",
"null"
],
"additionalProperties": true,
"properties": {
"recipe": {
"type": [
"string",
"null"
]
},
"recipes": {
"type": "array",
"items": {
"type": "object",
"properties": {
"key": {
"type": "string"
},
"label": {
"type": "string"
},
"rows": {
"type": "integer"
},
"tiered": {
"type": "integer"
}
},
"required": [
"key",
"label",
"rows",
"tiered"
]
}
}
}
}
},
"required": [
"view",
"apiVersion",
"params"
],
"additionalProperties": true
}Scroll to inspect the full example
Response example
{
"view": "cache-reuse",
"apiVersion": "v1",
"params": {}
}404
Source status is preserved; internal details are removed and failures are not cached.
Media type: application/jsonResponse shape
{
"type": "object",
"additionalProperties": true
}Response example
{
"error": "Source data unavailable"
}429
Source status is preserved; internal details are removed and failures are not cached.
Media type: application/jsonResponse shape
{
"type": "object",
"additionalProperties": true
}Response example
{
"error": "Source data unavailable"
}503
Source status is preserved; internal details are removed and failures are not cached.
Media type: application/jsonResponse shape
{
"type": "object",
"additionalProperties": true
}Response example
{
"error": "Source data unavailable"
}400
Invalid or unsupported query.
Media type: application/jsonResponse shape
{
"type": "object",
"additionalProperties": true
}Response example
{
"error": "Invalid parameter",
"param": "model"
}500
Data retrieval or projection failed.
Media type: application/jsonResponse shape
{
"type": "object",
"additionalProperties": true
}Response example
{
"error": "Internal server error"
}GETBeta/api/v1/views/profit-estimatorProfit per chip-hour
Read-only profit per chip-hour using the dashboard's source handlers and calculation helpers. Unknown and repeated query keys return 400. The response includes resolved params and preserves missing evidence. Renderer-only styling is not an API parameter.
Parameters
| Name | Location | Type | Requirement | Description | Example |
|---|---|---|---|---|---|
| cacheHitMode | query | string | Optional | actual or theoretical (default, displayed as Theoretical roofline). Actual uses the server-measured cache hit rate from the benchmark; theoretical uses the infinite-cache rate computed from the trace, falling back to the measured rate when unavailable. | |
| cachedInputPrice | query | string | Optional | Custom cached input USD/million tokens, default 0.1. | |
| costProvider | query | string | Optional | costh = owning, costr = renting. Profit views additionally accept custom with customCosts. No costn provider. | |
| customCosts | query | string | Optional | JSON object from base hardware keys to finite nonnegative USD/chip-hour values, at most 100 entries. | |
| date | query | string | Optional | Snapshot cutoff date, YYYY-MM-DD. runId selects an exact logical run snapshot instead. | |
| dates | query | string | Optional | Up to twelve comma-separated YYYY-MM-DD or YYYY-MM-DD~rRUN_ID comparison entries. Date-only entries select that exact logical snapshot, not an as-of cutoff; run entries select the exact logical run snapshot. Each snapshot is evaluated independently. | |
| end | query | string | Optional | Comparison range end, YYYY-MM-DD; must not precede start. Historical uses an inclusive data bound. | |
| gpus | query | string | Optional | Comma-separated hardware/config keys; for gpu-metrics use numeric GPU indices. Omit for all. | |
| inputPrice | query | string | Optional | Custom uncached input USD/million tokens, default 1. | |
| labCut | query | string | Optional | Model license/revenue-share percentage, 0 through 100; model-specific default. | |
| model | query | string | Required | Model display name or comparison slug. The dashboards offer agentic-trace models with usable benchmark curves and token pricing; this API also accepts other known models, which may return no estimated rows. Check /api/v1/availability for agentic_traces data. | DeepSeek-V4-Pro |
| outputPrice | query | string | Optional | Custom output USD/million tokens, default 1. | |
| percentile | query | string | Optional | AgentX latency percentile p75 or p90; CollectiveX supports p50, p95, p99. | |
| powerBasis | query | string | Optional | provisioned (default, All in Provisioned), modeled (All in Measured) or compare. For GW-year estimates, modeled filters source points to valid system-power inputs before building the curve at the same target, without extrapolation or historical substitution. compare uses identical valid-curve throughput for paired budgets; when no valid curve reaches the target, it retains the original provisioned estimate. provisioned-only keeps the original performance curve. All in Measured adds modeled components and PUE to measured inputs; it is not measured wall power. NVL72 rows need validated GPU telemetry; each compute tray takes an equal share of a modeled liquid-cooled rack, with measured Grace-socket power when the row has complete validated socket telemetry (CPU-rail-only and module-sensor readings are not accepted) and modeled Grace power otherwise. Both interpolation knots must share a power basis. powerSource records topology (chassis or nvl72-tray), cpu (measured or modeled), PUE and the pinned power-model revision. powerLabel marks paired estimates and full-chassis extrapolation; skipped reasons preserve missing coverage. | |
| precisions | query | string | Optional | Comma-separated precision keys; omitted selection uses available curve density. Calculator extensions auto-select the densest official precision and include precisions present in unofficial-run overlays. | |
| priceSource | query | string | Optional | Profit: list, openrouter or custom. Inference/history: normalized or openrouter. Returned pricing records the prices actually used. | |
| runId | query | string | Optional | Positive safe integer workflow run ID. Omit to select the default run; required for GPU metrics. | 123 |
| sequence | query | string | Optional | Workload: 1k/1k, 1k/8k, 8k/1k or agentic-traces. AgentX extension views default to agentic-traces. | |
| start | query | string | Optional | Comparison range start, YYYY-MM-DD. With end, adds the two endpoints, not every intermediate date. Historical uses an inclusive data bound. | |
| target | query | string | Optional | Positive operating-point target, tok/s/user in interactivity-to-throughput mode. Both profit views use model-specific defaults: Kimi K3 60, GLM-5.2/5.3 135, MiniMax M3 115, DeepSeek V4 Pro 170 and DeepSeek V4.1 Flash 150 tok/s/user. Dashboard links use /<model>/target/<value> route segments; this API continues to accept the target query parameter. These are editable SKU-coverage assumptions, not OpenRouter P90-equivalent measurements. | |
| tcoBasis | query | string | Optional | internal (default) or external. Applies the same configured owning/rental cost basis as the UI. | |
| unofficialrun | query | string | Optional | Up to eight comma-separated public CI run IDs. Surrounding whitespace and duplicates are removed; run indices follow the remaining order. Overlay sources remain separate from official data. | |
| utilization | query | string | Optional | Utilization percent from 0 through 100, default 60. |
Request
curl
curl -sS 'https://inferencex.semianalysis.com/api/v1/views/profit-estimator?model=DeepSeek-V4-Pro'Responses
ProfitEstimatorView200
Resolved selection and public view data.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"view": {
"type": "string",
"enum": [
"profit-estimator"
]
},
"apiVersion": {
"type": "string",
"enum": [
"v1"
]
},
"params": {
"type": "object",
"additionalProperties": true
},
"pricing": {
"type": "object",
"additionalProperties": true
},
"data": {
"type": "object",
"additionalProperties": true
},
"overlays": {
"type": "object",
"additionalProperties": true
},
"comparisons": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": true
}
}
},
"required": [
"view",
"apiVersion",
"params"
],
"additionalProperties": true
}Scroll to inspect the full example
Response example
{
"view": "profit-estimator",
"apiVersion": "v1",
"params": {}
}404
Source status is preserved; internal details are removed and failures are not cached.
Media type: application/jsonResponse shape
{
"type": "object",
"additionalProperties": true
}Response example
{
"error": "Source data unavailable"
}429
Source status is preserved; internal details are removed and failures are not cached.
Media type: application/jsonResponse shape
{
"type": "object",
"additionalProperties": true
}Response example
{
"error": "Source data unavailable"
}503
Source status is preserved; internal details are removed and failures are not cached.
Media type: application/jsonResponse shape
{
"type": "object",
"additionalProperties": true
}Response example
{
"error": "Source data unavailable"
}400
Invalid or unsupported query.
Media type: application/jsonResponse shape
{
"type": "object",
"additionalProperties": true
}Response example
{
"error": "Invalid parameter",
"param": "model"
}500
Data retrieval or projection failed.
Media type: application/jsonResponse shape
{
"type": "object",
"additionalProperties": true
}Response example
{
"error": "Internal server error"
}GETBeta/api/v1/views/profit-estimator-per-gigawattProfit per gigawatt-year
Read-only profit per gigawatt-year using the dashboard's source handlers and calculation helpers. Unknown and repeated query keys return 400. The response includes resolved params and preserves missing evidence. Renderer-only styling is not an API parameter.
Parameters
| Name | Location | Type | Requirement | Description | Example |
|---|---|---|---|---|---|
| cacheHitMode | query | string | Optional | actual or theoretical (default, displayed as Theoretical roofline). Actual uses the server-measured cache hit rate from the benchmark; theoretical uses the infinite-cache rate computed from the trace, falling back to the measured rate when unavailable. | |
| cachedInputPrice | query | string | Optional | Custom cached input USD/million tokens, default 0.1. | |
| costProvider | query | string | Optional | costh = owning, costr = renting. Profit views additionally accept custom with customCosts. No costn provider. | |
| customCosts | query | string | Optional | JSON object from base hardware keys to finite nonnegative USD/chip-hour values, at most 100 entries. | |
| date | query | string | Optional | Snapshot cutoff date, YYYY-MM-DD. runId selects an exact logical run snapshot instead. | |
| dates | query | string | Optional | Up to twelve comma-separated YYYY-MM-DD or YYYY-MM-DD~rRUN_ID comparison entries. Date-only entries select that exact logical snapshot, not an as-of cutoff; run entries select the exact logical run snapshot. Each snapshot is evaluated independently. | |
| end | query | string | Optional | Comparison range end, YYYY-MM-DD; must not precede start. Historical uses an inclusive data bound. | |
| gpus | query | string | Optional | Comma-separated hardware/config keys; for gpu-metrics use numeric GPU indices. Omit for all. | |
| inputPrice | query | string | Optional | Custom uncached input USD/million tokens, default 1. | |
| labCut | query | string | Optional | Model license/revenue-share percentage, 0 through 100; model-specific default. | |
| model | query | string | Required | Model display name or comparison slug. The dashboards offer agentic-trace models with usable benchmark curves and token pricing; this API also accepts other known models, which may return no estimated rows. Check /api/v1/availability for agentic_traces data. | DeepSeek-V4-Pro |
| outputPrice | query | string | Optional | Custom output USD/million tokens, default 1. | |
| percentile | query | string | Optional | AgentX latency percentile p75 or p90; CollectiveX supports p50, p95, p99. | |
| powerBasis | query | string | Optional | provisioned (default, All in Provisioned), modeled (All in Measured) or compare. For GW-year estimates, modeled filters source points to valid system-power inputs before building the curve at the same target, without extrapolation or historical substitution. compare uses identical valid-curve throughput for paired budgets; when no valid curve reaches the target, it retains the original provisioned estimate. provisioned-only keeps the original performance curve. All in Measured adds modeled components and PUE to measured inputs; it is not measured wall power. NVL72 rows need validated GPU telemetry; each compute tray takes an equal share of a modeled liquid-cooled rack, with measured Grace-socket power when the row has complete validated socket telemetry (CPU-rail-only and module-sensor readings are not accepted) and modeled Grace power otherwise. Both interpolation knots must share a power basis. powerSource records topology (chassis or nvl72-tray), cpu (measured or modeled), PUE and the pinned power-model revision. powerLabel marks paired estimates and full-chassis extrapolation; skipped reasons preserve missing coverage. | |
| precisions | query | string | Optional | Comma-separated precision keys; omitted selection uses available curve density. Calculator extensions auto-select the densest official precision and include precisions present in unofficial-run overlays. | |
| priceSource | query | string | Optional | Profit: list, openrouter or custom. Inference/history: normalized or openrouter. Returned pricing records the prices actually used. | |
| runId | query | string | Optional | Positive safe integer workflow run ID. Omit to select the default run; required for GPU metrics. | 123 |
| sequence | query | string | Optional | Workload: 1k/1k, 1k/8k, 8k/1k or agentic-traces. AgentX extension views default to agentic-traces. | |
| start | query | string | Optional | Comparison range start, YYYY-MM-DD. With end, adds the two endpoints, not every intermediate date. Historical uses an inclusive data bound. | |
| target | query | string | Optional | Positive operating-point target, tok/s/user in interactivity-to-throughput mode. Both profit views use model-specific defaults: Kimi K3 60, GLM-5.2/5.3 135, MiniMax M3 115, DeepSeek V4 Pro 170 and DeepSeek V4.1 Flash 150 tok/s/user. Dashboard links use /<model>/target/<value> route segments; this API continues to accept the target query parameter. These are editable SKU-coverage assumptions, not OpenRouter P90-equivalent measurements. | |
| tcoBasis | query | string | Optional | internal (default) or external. Applies the same configured owning/rental cost basis as the UI. | |
| unofficialrun | query | string | Optional | Up to eight comma-separated public CI run IDs. Surrounding whitespace and duplicates are removed; run indices follow the remaining order. Overlay sources remain separate from official data. | |
| utilization | query | string | Optional | Utilization percent from 0 through 100, default 60. |
Request
curl
curl -sS 'https://inferencex.semianalysis.com/api/v1/views/profit-estimator-per-gigawatt?model=DeepSeek-V4-Pro'Responses
ProfitEstimatorPerGigawattView200
Resolved selection and public view data.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"view": {
"type": "string",
"enum": [
"profit-estimator-per-gigawatt"
]
},
"apiVersion": {
"type": "string",
"enum": [
"v1"
]
},
"params": {
"type": "object",
"additionalProperties": true
},
"pricing": {
"type": "object",
"additionalProperties": true
},
"data": {
"type": "object",
"additionalProperties": true
},
"overlays": {
"type": "object",
"additionalProperties": true
},
"comparisons": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": true
}
}
},
"required": [
"view",
"apiVersion",
"params"
],
"additionalProperties": true
}Scroll to inspect the full example
Response example
{
"view": "profit-estimator-per-gigawatt",
"apiVersion": "v1",
"params": {}
}404
Source status is preserved; internal details are removed and failures are not cached.
Media type: application/jsonResponse shape
{
"type": "object",
"additionalProperties": true
}Response example
{
"error": "Source data unavailable"
}429
Source status is preserved; internal details are removed and failures are not cached.
Media type: application/jsonResponse shape
{
"type": "object",
"additionalProperties": true
}Response example
{
"error": "Source data unavailable"
}503
Source status is preserved; internal details are removed and failures are not cached.
Media type: application/jsonResponse shape
{
"type": "object",
"additionalProperties": true
}Response example
{
"error": "Source data unavailable"
}400
Invalid or unsupported query.
Media type: application/jsonResponse shape
{
"type": "object",
"additionalProperties": true
}Response example
{
"error": "Invalid parameter",
"param": "model"
}500
Data retrieval or projection failed.
Media type: application/jsonResponse shape
{
"type": "object",
"additionalProperties": true
}Response example
{
"error": "Internal server error"
}GETBeta/api/v1/views/collectivexCollectiveX EP, KV and swap views
Read-only collectivex ep, kv and swap views using the dashboard's source handlers and calculation helpers. Unknown and repeated query keys return 400. The response includes resolved params and preserves missing evidence. Renderer-only styling is not an API parameter.
Parameters
| Name | Location | Type | Requirement | Description | Example |
|---|---|---|---|---|---|
| activeSeries | query | string | Optional | Comma-separated run-namespaced EP series IDs. Omit for all. | |
| kvSeries | query | string | Optional | Comma-separated run:case IDs for KV series. Omit for all. | |
| swapSeries | query | string | Optional | Comma-separated swap series IDs. Omit for all. | |
| backend | query | string | Optional | Backend filter; omit for all. | |
| epSize | query | string | Optional | EP size, default 8 when available, otherwise the first available size. | |
| kvOp | query | string | Optional | KV operation: push (default) or pull. | |
| kvX | query | string | Optional | KV projection: isl (default), batch, frontier or overlap. | |
| kvY | query | string | Optional | KV metric: latency (default) or bandwidth. | |
| modes | query | string | Optional | Comma-separated EP mode keys, default every available mode. | |
| operation | query | string | Optional | EP operation: pair_period (default when the run carries it, else roundtrip; the chained steady-state period), roundtrip (fresh entry), dispatch or combine. | |
| overlapIsl | query | string | Optional | Overlap input length: auto (default; the ISL with the widest batch sweep) or a positive integer. | |
| pageTokens | query | string | Optional | Deprecated and ignored: KV rows move whole cache rows, with no page size. | |
| percentile | query | string | Optional | AgentX latency percentile p75 or p90; CollectiveX supports p50, p95, p99. | |
| phase | query | string | Optional | CollectiveX phase, default decode; video phase: measurement, startup or warmup. | |
| precision | query | string | Optional | Precision key; CollectiveX defaults to fp8 if available. The image view trims whitespace, ignores case and defaults to all. | |
| runs | query | string | Optional | Ordered comma-separated run IDs, at most eight. Omit for newest measured run; empty value selects none. | |
| sku | query | string | Optional | CollectiveX chip SKU filter; default all. | |
| suite | query | string | Optional | Run-list filter: all, ep, kv or swap. | |
| swapDirection | query | string | Optional | h2d (default), d2h or d2d. | |
| swapLayout | query | string | Optional | contiguous (default) or random. | |
| swapMetric | query | string | Optional | bandwidth (default) or latency. | |
| swapPercentile | query | string | Optional | p50 (default), p95 or p99. | |
| version | query | string | Optional | CollectiveX dataset schema version; default 1. | |
| yAxis | query | string | Optional | CollectiveX: latency, tokens-per-second, activation-rate or payload-rate. Video: dollar, clipsGpu, secondsGpu, clipsAllocatedGpu, secondsAllocatedGpu or energy. |
Request
curl
curl -sS 'https://inferencex.semianalysis.com/api/v1/views/collectivex'Responses
CollectivexView200
Resolved selection and public view data.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"view": {
"type": "string",
"enum": [
"collectivex"
]
},
"apiVersion": {
"type": "string",
"enum": [
"v1"
]
},
"params": {
"type": "object",
"additionalProperties": true
},
"runs": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": true
}
},
"ep": {
"type": "object",
"additionalProperties": true
},
"kv": {
"type": "object",
"additionalProperties": true
},
"swap": {
"type": "object",
"additionalProperties": true
},
"options": {
"type": "object",
"additionalProperties": true
},
"knownSupport": {
"type": "object",
"additionalProperties": true
},
"knownSupportFootnotes": {
"type": "object",
"additionalProperties": true
}
},
"required": [
"view",
"apiVersion",
"params"
],
"additionalProperties": true
}Scroll to inspect the full example
Response example
{
"view": "collectivex",
"apiVersion": "v1",
"params": {}
}404
Source status is preserved; internal details are removed and failures are not cached.
Media type: application/jsonResponse shape
{
"type": "object",
"additionalProperties": true
}Response example
{
"error": "Source data unavailable"
}429
Source status is preserved; internal details are removed and failures are not cached.
Media type: application/jsonResponse shape
{
"type": "object",
"additionalProperties": true
}Response example
{
"error": "Source data unavailable"
}503
Source status is preserved; internal details are removed and failures are not cached.
Media type: application/jsonResponse shape
{
"type": "object",
"additionalProperties": true
}Response example
{
"error": "Source data unavailable"
}400
Invalid or unsupported query.
Media type: application/jsonResponse shape
{
"type": "object",
"additionalProperties": true
}Response example
{
"error": "Invalid parameter",
"param": "model"
}500
Data retrieval or projection failed.
Media type: application/jsonResponse shape
{
"type": "object",
"additionalProperties": true
}Response example
{
"error": "Internal server error"
}GETBeta/api/v1/views/submissionsSubmission table and weekly volume
Read-only submission table and weekly volume using the dashboard's source handlers and calculation helpers. Unknown and repeated query keys return 400. The response includes resolved params and preserves missing evidence. Renderer-only styling is not an API parameter.
Parameters
| Name | Location | Type | Requirement | Description | Example |
|---|---|---|---|---|---|
| direction | query | string | Optional | asc or desc. Defaults: submissions desc, GPU statistics asc. | |
| limit | query | string | Optional | Submission table row limit, default 100, maximum 10000. | |
| lines | query | string | Optional | Submission chart lines, comma-separated from nvidia, amd, openai, google, huawei, unknown and total. Defaults to every vendor with submissions plus total, as in the UI. | |
| mode | query | string | Optional | Submissions: weekly (default) or cumulative. Calculator modes use interactivity_to_throughput or throughput_to_interactivity. | |
| offset | query | string | Optional | Submission table offset, default 0. | |
| onChangeOnly | query | string | Optional | Boolean, default true. For weekly charts only, apply the dashboard on-change reporting cutoff. | |
| search | query | string | Optional | Case-insensitive submission search over chip, model, framework, precision, speculation and vendor. | |
| sort | query | string | Optional | Table sort column; submissions defaults to date, GPU statistics to gpuIndex. |
Request
curl
curl -sS 'https://inferencex.semianalysis.com/api/v1/views/submissions'Responses
SubmissionsView200
Resolved selection and public view data.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"view": {
"type": "string",
"enum": [
"submissions"
]
},
"apiVersion": {
"type": "string",
"enum": [
"v1"
]
},
"params": {
"type": "object",
"additionalProperties": true
},
"rows": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": true
}
},
"stats": {
"type": "object",
"additionalProperties": true
},
"series": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": true
}
}
},
"required": [
"view",
"apiVersion",
"params"
],
"additionalProperties": true
}Scroll to inspect the full example
Response example
{
"view": "submissions",
"apiVersion": "v1",
"params": {}
}404
Source status is preserved; internal details are removed and failures are not cached.
Media type: application/jsonResponse shape
{
"type": "object",
"additionalProperties": true
}Response example
{
"error": "Source data unavailable"
}429
Source status is preserved; internal details are removed and failures are not cached.
Media type: application/jsonResponse shape
{
"type": "object",
"additionalProperties": true
}Response example
{
"error": "Source data unavailable"
}503
Source status is preserved; internal details are removed and failures are not cached.
Media type: application/jsonResponse shape
{
"type": "object",
"additionalProperties": true
}Response example
{
"error": "Source data unavailable"
}400
Invalid or unsupported query.
Media type: application/jsonResponse shape
{
"type": "object",
"additionalProperties": true
}Response example
{
"error": "Invalid parameter",
"param": "model"
}500
Data retrieval or projection failed.
Media type: application/jsonResponse shape
{
"type": "object",
"additionalProperties": true
}Response example
{
"error": "Internal server error"
}GETBeta/api/v1/views/current-inferencex-imageCurrent serving images
Read-only current serving images using the dashboard's source handlers and calculation helpers. Unknown and repeated query keys return 400. The response includes resolved params and preserves missing evidence. Renderer-only styling is not an API parameter. ageSummary is the nearest-rank p90 of current filtered row submission ages at asOf (future-dated rows excluded). includeHistory=true adds 90 daily ageHistory points reconstructed from the successful image-bearing submissions of those same configurations, so the last point equals ageSummary. Configurations absent from the current table, such as speculation or node-type variants replaced by a newer run, are excluded. Each displayed configuration has one vote; repeated submissions reset its age even if the image tag is unchanged. Neither metric measures container build age. A row is outdated when its tag version is below the base engine's latest vLLM or SGLang release. A nightly whose commit GitHub resolves is outdated only when that commit is an ancestor of the release. Other nightlies compare their build date with the release's publish date: the tag's date, else the build's earliest submission in the catalog. Other tags, and engines without a release stream, are not flagged.
Parameters
| Name | Location | Type | Requirement | Description | Example |
|---|---|---|---|---|---|
| asOf | query | string | Optional | Reference date YYYY-MM-DD for reproducible rolling reliability or image-age calculations. | |
| includeHistory | query | string | Optional | true or false (default). Include 90 daily UTC image-age points ending on asOf, with nearest-rank p90AgeDays and configurationCount. Covers the same configurations as ageSummary (current filtered rows dated on or before asOf), so the last point equals ageSummary. Each configuration replays its successful image-bearing submissions up to its current row date, with an older seed before the window. Missing p90 is null; no future observations are backfilled. All image filters apply. This is submission age, not container build age. | |
| frameworks | query | string | Optional | Comma-separated framework-family keys; omitted selection includes all. | |
| hardware | query | string | Optional | Image-view hardware key; whitespace is trimmed and case ignored. Default all. | |
| model | query | string | Optional | Model display name or comparison slug, case-insensitive. Required for benchmark-based views. The image view also accepts current image-catalog names, trims whitespace and defaults to all; unknown models return 400. | DeepSeek-V4-Pro |
| nodeType | query | string | Optional | Image deployment: all (default), single or disagg. | |
| precision | query | string | Optional | Precision key; CollectiveX defaults to fp8 if available. The image view trims whitespace, ignores case and defaults to all. | |
| sequence | query | string | Optional | Image scenario. Defaults to agentic-traces (Agentic), falling back to the first available active scenario for the selected model. | |
| spec | query | string | Optional | Speculative decoding filter; the image view trims whitespace and ignores case for a single value, default all. Inference uses a comma-separated list. |
Request
curl
curl -sS 'https://inferencex.semianalysis.com/api/v1/views/current-inferencex-image'Responses
CurrentInferencexImageView200
Resolved selection and public view data.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"view": {
"type": "string",
"enum": [
"current-inferencex-image"
]
},
"apiVersion": {
"type": "string",
"enum": [
"v1"
]
},
"params": {
"type": "object",
"additionalProperties": true
},
"rows": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": true
}
},
"options": {
"type": "object",
"additionalProperties": true
},
"releases": {
"type": "object",
"additionalProperties": true
},
"ageSummary": {
"type": "object",
"properties": {
"date": {
"type": "string",
"format": "date"
},
"p90AgeDays": {
"type": [
"integer",
"null"
],
"minimum": 0
},
"configurationCount": {
"type": "integer",
"minimum": 0
}
},
"required": [
"date",
"p90AgeDays",
"configurationCount"
],
"additionalProperties": false
},
"ageHistory": {
"type": "array",
"items": {
"type": "object",
"properties": {
"date": {
"type": "string",
"format": "date"
},
"p90AgeDays": {
"type": [
"integer",
"null"
],
"minimum": 0
},
"configurationCount": {
"type": "integer",
"minimum": 0
}
},
"required": [
"date",
"p90AgeDays",
"configurationCount"
],
"additionalProperties": false
}
},
"ageMethodology": {
"type": "object",
"additionalProperties": true
}
},
"required": [
"view",
"apiVersion",
"params"
],
"additionalProperties": true
}Scroll to inspect the full example
Response example
{
"view": "current-inferencex-image",
"apiVersion": "v1",
"params": {}
}404
Source status is preserved; internal details are removed and failures are not cached.
Media type: application/jsonResponse shape
{
"type": "object",
"additionalProperties": true
}Response example
{
"error": "Source data unavailable"
}429
Source status is preserved; internal details are removed and failures are not cached.
Media type: application/jsonResponse shape
{
"type": "object",
"additionalProperties": true
}Response example
{
"error": "Source data unavailable"
}503
Source status is preserved; internal details are removed and failures are not cached.
Media type: application/jsonResponse shape
{
"type": "object",
"additionalProperties": true
}Response example
{
"error": "Source data unavailable"
}400
Invalid or unsupported query.
Media type: application/jsonResponse shape
{
"type": "object",
"additionalProperties": true
}Response example
{
"error": "Invalid parameter",
"param": "model"
}500
Data retrieval or projection failed.
Media type: application/jsonResponse shape
{
"type": "object",
"additionalProperties": true
}Response example
{
"error": "Internal server error"
}GETBeta/api/v1/views/gpu-metricsGPU telemetry and full-record statistics
Read-only gpu telemetry and full-record statistics using the dashboard's source handlers and calculation helpers. Unknown and repeated query keys return 400. The response includes resolved params and preserves missing evidence. Renderer-only styling is not an API parameter. Telemetry comes from one source per run: its stored DB series once the run is ingested, otherwise its GitHub artifacts. Stored coverage describes the entire run: complete means the recorded artifact/file inventory is present, partial means recorded telemetry is missing, and unknown means no complete inventory was retained; missingArtifacts names entirely missing artifacts and may be empty when only a file is incomplete. GitHub coverage is null (unassessed). Responses use private, no-store; an upstream database failure remains HTTP 503 rather than an empty dataset. File/host series retain separate identities. Raw rows and charts respect selected GPU indices. Statistics cover every chip of the selected file/host series, including startup and warmup, and are computed from its samples on each read; a stored series whose samples do not match its recorded sample count returns none. Mean is sample-weighted, P50/P95/P99 interpolate linearly at p*(N-1), and standard deviation divides by N. These statistics are not serving-window power or J/token. Line time remains relative to the first sample across all chips; missing metric readings are omitted rather than zero-filled. Correlations require both readings. This endpoint projects the raw GPU-metrics explorer, not the inference Power Timeline. Timeline role/window evidence comes from its existing series=power read plus benchmark power_audit. Timeline i_ptaxis, i_ptlines, i_ptwindow, i_ptfocus, i_ptutility and i_ptconc share fields control client rendering only and are not accepted here; they do not change raw telemetry or full-record statistics.
Parameters
| Name | Location | Type | Requirement | Description | Example |
|---|---|---|---|---|---|
| artifact | query | string | Optional | GPU metric artifact name, or <artifact>/<file> for multi-file artifacts, as returned in artifacts[]. For video, a numeric artifact ID paired with run. | |
| chartView | query | string | Optional | chart (default) or correlation. Both return unsampled source rows. | |
| corrXMetric | query | string | Optional | Correlation x-axis GPU metric, default power. | |
| corrYMetric | query | string | Optional | Correlation y-axis GPU metric. Defaults to temperature when the artifact sampled it, otherwise to the first collected metric other than corrXMetric; without one the correlation is empty. An explicit metric is used even when unsampled, giving an empty correlation. | |
| direction | query | string | Optional | asc or desc. Defaults: submissions desc, GPU statistics asc. | |
| downsample | query | string | Optional | Boolean, default true; sets the renderer hint maxInteractivePoints to 2000, or null when false. Returned raw rows and statistics are never sampled. | |
| gpus | query | string | Optional | Comma-separated hardware/config keys; for gpu-metrics use numeric GPU indices. Omit for all. | |
| metric | query | string | Optional | View-specific metric key. GPU metrics: power, temperature, clocks, utilization or available AMD metrics. | |
| runId | query | string | Required | Positive safe integer workflow run ID. Omit to select the default run; required for GPU metrics. | 123 |
| sort | query | string | Optional | Table sort column; submissions defaults to date, GPU statistics to gpuIndex. |
Request
curl
curl -sS 'https://inferencex.semianalysis.com/api/v1/views/gpu-metrics?runId=123'Responses
GpuMetricsView200
Resolved selection and public view data.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"view": {
"type": "string",
"enum": [
"gpu-metrics"
]
},
"apiVersion": {
"type": "string",
"enum": [
"v1"
]
},
"params": {
"type": "object",
"additionalProperties": true
},
"runInfo": {
"type": "object",
"additionalProperties": true
},
"artifacts": {
"type": "array",
"items": {
"type": "string"
}
},
"coverage": {
"type": [
"object",
"null"
],
"properties": {
"status": {
"type": "string",
"enum": [
"complete",
"partial",
"unknown"
]
},
"missingArtifacts": {
"type": "array",
"items": {
"type": "string"
}
}
},
"required": [
"status",
"missingArtifacts"
],
"additionalProperties": false
},
"rows": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": true
}
},
"stats": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": true
},
"description": "Per-GPU statistics for the selected file/host series, including startup and warmup. For stored runs they are computed on each read from the stored samples: a missing reading stays missing, and a series whose stored samples do not match its recorded sample count returns no statistics. Count is the finite-reading count after first-wins timestamp/GPU deduplication. Mean is sample-weighted, P50/P95/P99 use linear interpolation at p*(N-1), and standard deviation divides by N. Values use the selected metric unit; rows carry no metric field, since params.metric identifies it. GPU visibility and chart downsampling do not change this population. These are not serving-window power or J/token."
},
"rendering": {
"type": "object",
"additionalProperties": true
}
},
"required": [
"view",
"apiVersion",
"params"
],
"additionalProperties": true
}Scroll to inspect the full example
Response example
{
"view": "gpu-metrics",
"apiVersion": "v1",
"params": {}
}404
Source status is preserved; internal details are removed and failures are not cached.
Media type: application/jsonResponse shape
{
"type": "object",
"additionalProperties": true
}Response example
{
"error": "Source data unavailable"
}429
Source status is preserved; internal details are removed and failures are not cached.
Media type: application/jsonResponse shape
{
"type": "object",
"additionalProperties": true
}Response example
{
"error": "Source data unavailable"
}503
Source status is preserved; internal details are removed and failures are not cached.
Media type: application/jsonResponse shape
{
"type": "object",
"additionalProperties": true
}Response example
{
"error": "Source data unavailable"
}400
Invalid or unsupported query.
Media type: application/jsonResponse shape
{
"type": "object",
"additionalProperties": true
}Response example
{
"error": "Invalid parameter",
"param": "model"
}500
Data retrieval or projection failed.
Media type: application/jsonResponse shape
{
"type": "object",
"additionalProperties": true
}Response example
{
"error": "Internal server error"
}GETBeta/api/v1/views/videoVideoGenX dashboard and published comparison evidence
view=overview shares resultProjection with the /vgx results UI, admits one canonical 20-clip C1 workload with ten valid samples per hardware, and returns measured metrics plus separately labelled participating/allocated cost and modelled profit. Missing allocation remains null. point.videoSecondsPerHour carries validated decoded output seconds per measurement wall hour from valid measured requests, excluding warmup. Video-second throughput, both per-video-second cost bases and modelled revenue use this output rate. Missing valid output durations or older History without this optional field leave these metrics and profit null; requested durationSeconds remains workload/filter metadata and is never a fallback. Default points require publication approval. unofficialrun selects only the exact run into overlayCells (selected GPU count) and overlayDeploymentCells (all matching GPU counts), excluding it from official cells, deployments and workload mix. A stored run keeps its actual approval; a run absent from History is read live from its public h3-results artifact documents, unapproved and without media; its clips still carry PSNR and unscored reasons from a trusted h3-quality artifact of the same run attempt. Approved cohorts retain default preference; named-only workloads require explicit selection or no matching approved cohort. Clearing never promotes unapproved evidence. Dismissal retains a workload selector only if it remains in the approved choices. The overview returns JSON by default; format=csv returns selected official and named cells with source identities, units and both GPU cost bases as a private no-store download. duration_s is the requested plan duration; video_seconds_per_s beside clips_per_s is validated decoded output seconds per wall second. Missing values are empty CSV cells. Missing quality, stage timing, memory and startup evidence stays null. Play/pause, scrub, difference rendering, theme and expand controls are presentation-only; media uses the published-only read path. Read-only videogenx dashboard and published comparison evidence using the dashboard's source handlers and calculation helpers. Unknown and repeated query keys return 400. The response includes resolved params and preserves missing evidence. Renderer-only styling is not an API parameter. Default dashboard mode reuses published history → videoPoints → dashboardCells → metrics/plot/compare/evidence. It reads at most the same first five history pages as the UI, reports coverage/truncation, selects one canonical workload and keeps the shared sample floor. v_hidden affects chart/table only. view=compare additionally reads at most two already-published media artifacts, pairs cases with the UI helper and returns relative media paths without asset URLs; missing/failed media remains explicit. run/artifact or explicit discovery/results/tradeoff preserve the legacy contract. Discovery includes standard run-sweep runs; these may have no video artifacts. Run artifact listings cover all GitHub pages and retain legacy and unique native matrix names. Allocated GPU counts require retained scheduler allocation evidence; native power device counts cannot substitute for it. Missing allocation and allocation-normalized metrics remain null. No date range, full-catalog completeness, power/quality qualification or causal bottleneck claim is implied. Local bundles and arbitrary URLs are excluded. HTTP responses remain private, no-store. Published History and media metadata are cached across instances under the video tag, without a GitHub read. Errors, History pages whose index or approval reads failed and missing media are not retained; publication, History backfill or ?scope=video expires that cache at once. Server page seeding and first-frame preloading are presentation-only and change no selectors, metrics or source membership.
Parameters
| Name | Location | Type | Requirement | Description | Example |
|---|---|---|---|---|---|
| r_model | query | string | Optional | Exact model identifier; omitted selects the best-covered eligible workload. | |
| r_workload | query | string | Optional | Compact key from options.workloads; resolves to the complete canonical workloadKey internally. Any source point ID of the same workload also resolves, so shared links survive later publications. One shared 20-scheduled-clip C1 spec with at least 10 valid samples per hardware. | |
| r_resolution | query | string | Optional | Exact width×height from the sealed plan, such as 1344×768. | |
| r_duration | query | string | Optional | Positive requested clip duration from the plan, in seconds; not decoded output duration. | |
| r_mode | query | string | Optional | Explicit generation mode; unknown modes are never inferred from model names. | |
| r_steps | query | string | Optional | Recorded denoising step count. | |
| r_gpus | query | string | Optional | Deprecated overview control, accepted but ignored. All GPU counts are included; params.gpuCount is null. | |
| r_engine | query | string | Optional | Recorded engine name, separate from runtime revision. | |
| r_date | query | string | Optional | As-of execution date, YYYY-MM-DD; never publication date. Each official deployment shows its latest run measured on or before this date; named-run overlays are not filtered. runs lists every official run of the workload and run names the selected one. | |
| r_run | query | string | Optional | GitHub Actions run ID on the r_date day to view as of; default is that day’s last run. | |
| r_configs | query | string | Optional | Comma-separated chip configs (hardware:GPUs:tp:ulysses keys from chipConfigs, at most 4) whose official runs history returns for the Compare history chart. | |
| r_from | query | string | Optional | Earliest execution date, YYYY-MM-DD, of the runs history returns; default unbounded. | |
| r_to | query | string | Optional | Latest execution date, YYYY-MM-DD, of the runs history returns; default unbounded. | |
| r_hidden | query | string | Optional | Comma-separated hardware keys hidden from the overview chart, table, deployment detail and clips. | |
| r_deadline | query | string | Optional | Delivery deadline seconds, default 120. Attainment divides on-time valid clips by valid clips; goodput divides by measured wall time. | |
| r_tier | query | string | Optional | Hardware registry TCO tier h (owning, default) or r (renting). Both GPU cost bases are returned. | |
| r_api | query | string | Optional | Optional nonnegative USD/video-second scenario price when generation mode is unknown; default null. | |
| r_price_t2v | query | string | Optional | Optional nonnegative T2V scenario price, USD/video-second; default null. | |
| r_price_i2v | query | string | Optional | Optional nonnegative I2V scenario price, USD/video-second; default null. | |
| r_price_r2v | query | string | Optional | Optional nonnegative R2V scenario price, USD/video-second; default null. | |
| r_utilization | query | string | Optional | Modelled utilization 0–1, default 1. Revenue scales; reserved GPU hourly cost does not. | |
| r_x | query | string | Optional | Deprecated overview control, accepted but ignored. The chart always uses median per-clip latency; params.xMetric is p50. | |
| r_y | query | string | Optional | Chart y metric: videoSecondsPerDollar (default), videosPerGpuHour or kjPerClip. Presentation only; invalid values use the default. | |
| r_basis | query | string | Optional | Deprecated overview control, accepted but ignored. The chart always uses participating GPUs; params.costBasis is participating. Both cost bases remain in the evidence and CSV. | |
| r_optimal | query | string | Optional | Optimal Only: 0 also plots dominated deployments; any other value plots each hardware frontier only. Presentation only. | |
| r_point | query | string | Optional | Open deployment detail as runId:pointId. Unknown or filtered-out keys show the overview. Presentation only. | |
| unofficialrun | query | string | Optional | Single run ID (unofficialrun or canonical unofficialruns alias; multiple IDs and both aliases together return 400) for separate overview overlayCells and overlayDeploymentCells. A stored run keeps its actual approval. A run absent from the five History pages is read live from the JSON documents of its public native h3-results artifacts and is always unapproved, with publishedAt null and no media. The selected run is excluded from official cells. Unknown, unreadable or ineligible runs return no overlay. No storage or publication is triggered. | |
| unofficialruns | query | string | Optional | Single run ID (unofficialrun or canonical unofficialruns alias; multiple IDs and both aliases together return 400) for separate overview overlayCells and overlayDeploymentCells. A stored run keeps its actual approval. A run absent from the five History pages is read live from the JSON documents of its public native h3-results artifacts and is always unapproved, with publishedAt null and no media. The selected run is excluded from official cells. Unknown, unreadable or ineligible runs return no overlay. No storage or publication is triggered. | |
| artifact | query | string | Optional | GPU metric artifact name, or <artifact>/<file> for multi-file artifacts, as returned in artifacts[]. For video, a numeric artifact ID paired with run. | |
| cell | query | string | Optional | Serving cell ID. Unknown IDs fall back to the first cell, matching the UI; evidence.cell records the resolved choice. | |
| compare | query | string | Optional | Video: up to eight comma-separated run:artifact pairs of already published evidence. | |
| costs | query | string | Optional | JSON object mapping video point IDs to {hourly,source,date} strings; hourly is nonnegative USD/deployment-hour. | |
| format | query | string | Optional | JSON by default. CSV is available only with view=overview and flattens its selected official and named cells with source identities, both GPU cost bases and units. duration_s is the requested plan duration; video_seconds_per_s is validated decoded output seconds per wall second. Empty CSV cells represent missing values. | |
| gpuBasis | query | string | Optional | Video denominator: participating (default) or allocated GPUs. | |
| page | query | string | Optional | Video one-based CI discovery page (view=discovery). Video dashboard/compare read at most the first five published history pages. | |
| phase | query | string | Optional | CollectiveX phase, default decode; video phase: measurement, startup or warmup. | |
| run | query | string | Optional | Public video CI run ID; retains legacy run/artifact selection. Omit for the dashboard, or use view=discovery to list runs. | |
| selected | query | string | Optional | Selected video tradeoff point ID. | |
| slot | query | string | Optional | Media/fidelity slot ID. Unknown IDs fall back to the UI default; evidence.slot records the resolved choice. | |
| source | query | string | Optional | Published video source ID; defaults to first source. | |
| v_x | query | string | Optional | Dashboard X metric: p90Latency (default) or p50Latency, in seconds. Invalid values use the UI default. | |
| v_y | query | string | Optional | Dashboard Y metric: videosPerDollar (default), dollarsPerVideo, videosPerGpuHour or kjPerVideo. Invalid values use the UI default. | |
| v_tier | query | string | Optional | TCO cost tier: h (owning, default) or r (renting), from the same hardware registry as the dashboard; invalid values use h. | |
| v_view | query | string | Optional | chart (default) or table; presentation-only preference, echoed as params.displayView without changing returned datasets. | |
| v_optimal | query | string | Optional | Per-hardware Pareto deployments only unless exactly 0. Changes chart/table rows; queued cells never enter either. | |
| v_api | query | string | Optional | Positive USD/video-second API list-price reference, rounded to four decimals; invalid values use the dated UI reference. Multiplies clip duration only, not TCO or chart efficiency. | |
| v_hidden | query | string | Optional | Comma-separated hidden hardware keys from the video roster; default empty, deduplicated and sorted, unknown keys ignored. Filters chart/table only, not KPI/Compare/Evidence. | |
| v_base | query | string | Optional | Baseline hardware key; unavailable/invalid selection resolves to the UI default (slowest eligible lead deployment). The two sides stay distinct: a side that would match the other moves to the other default. | |
| v_cand | query | string | Optional | Candidate hardware key; unavailable/invalid selection resolves to the UI default (fastest eligible lead deployment). The two sides stay distinct: a side that would match the other moves to the other default. | |
| v_case | query | string | Optional | Zero-based matched-case index, 0–9999 (default 0). view=compare clamps it to the available pairs and returns cases.resolvedCaseIndex; dashboard leaves media not-requested. | |
| view | query | string | Optional | Video overview (chart, results table, deployment detail; r_* selectors), dashboard (default without run/artifact), compare (published matched cases), or legacy discovery/results/tradeoff. run/artifact defaults to results. Selector families cannot be mixed. | |
| workload | query | string | Optional | Exact workload-group key from video workloads. Comparisons never pool unequal workloads. | |
| xAxis | query | string | Optional | Video latency: p90 or median. Serving defaults to median only when p90 is unavailable. | |
| yAxis | query | string | Optional | CollectiveX: latency, tokens-per-second, activation-rate or payload-rate. Video: dollar, clipsGpu, secondsGpu, clipsAllocatedGpu, secondsAllocatedGpu or energy. |
Request
curl
curl -sS 'https://inferencex.semianalysis.com/api/v1/views/video'Responses
VideoView200
Resolved selection and public view data.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"view": {
"type": "string",
"enum": [
"video"
]
},
"apiVersion": {
"type": "string",
"enum": [
"v1"
]
},
"params": {
"type": "object",
"additionalProperties": true
},
"cells": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": true
}
},
"deploymentCells": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": true
}
},
"workloadMix": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": true
}
},
"overlayCells": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": true
}
},
"overlayDeploymentCells": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": true
}
},
"clipQuality": {
"type": "object",
"description": "Per-clip evidence from the same selected artifacts and matching helper as the UI. No cohort PSNR aggregate; missing pairs remain null.",
"x-description-zh": "与界面共用所选产物和匹配函数的逐条视频证据。不计算样本组 PSNR 汇总值；缺失配对保留 null。",
"properties": {
"status": {
"type": "string",
"enum": [
"ready",
"partial",
"unavailable"
]
},
"groups": {
"type": "array",
"items": {
"type": "object",
"properties": {
"key": {
"type": "string"
},
"scope": {
"type": "string",
"enum": [
"ci",
"matched-noise-diagnostic"
]
},
"prompt": {
"type": "string"
},
"seed": {
"type": "number"
},
"take": {
"type": [
"integer",
"null"
]
},
"clips": {
"type": "array",
"items": {
"type": "object",
"properties": {
"hardware": {
"type": "string"
},
"pointId": {
"type": [
"string",
"null"
],
"description": "Performance point identity for CI clips; null for diagnostic reruns."
},
"contextPointId": {
"type": [
"string",
"null"
],
"description": "Original CI context for a diagnostic, never its media or performance identity."
},
"slotId": {
"type": [
"string",
"null"
]
},
"caseId": {
"type": [
"string",
"null"
]
},
"mediaPath": {
"type": [
"string",
"null"
]
},
"quality": {
"type": [
"object",
"null"
],
"properties": {
"attachmentSha256": {
"type": "string"
},
"scope": {
"type": "string",
"enum": [
"matched-noise-diagnostic"
]
},
"origin": {
"type": "object",
"additionalProperties": true,
"description": "Exact original CI candidate identity retained only as diagnostic context."
},
"candidate": {
"type": "object",
"additionalProperties": true,
"description": "Exact CI candidate identity, or the separately sealed native diagnostic run/receipt/media identity with captured-noise and topology metadata. Diagnostic clips do not inherit CI latency, energy or cost."
},
"reference": {
"type": "object",
"additionalProperties": true,
"description": "Reference case from the public descriptor: receipt/plan/media hashes, official Diffusers/model revisions, denoising_updates, sigma_points, and the stored MP4 url (empty for an unstored named run) with optional previewUrl. Reference execution is not a hardware performance result."
},
"scorer": {
"type": "object",
"additionalProperties": true,
"description": "evaluator.media.compare_media source revision and SHA256; method decoded-rgb24-all-frames."
},
"status": {
"type": "string",
"enum": [
"measured",
"identical",
"incompatible"
]
},
"psnrDb": {
"type": [
"number",
"null"
],
"description": "Decoded RGB24, every pixel/frame, peak=1 after division by 255, full-stream MSE; dB. Null for exact decoded equality or incompatible pairs; status distinguishes them. No quality pass/fail threshold."
},
"comparedFrames": {
"type": "integer",
"minimum": 0
},
"totalFrames": {
"type": "integer",
"minimum": 0
},
"checks": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": true
}
},
"notes": {
"type": "array",
"items": {
"type": "string"
}
},
"qualityStatus": {
"type": "string",
"enum": [
"unjudged"
]
},
"releaseQualified": {
"type": "boolean",
"enum": [
false
]
}
},
"required": [
"candidate",
"reference",
"scorer",
"status",
"psnrDb",
"comparedFrames",
"totalFrames",
"qualityStatus",
"releaseQualified"
]
},
"unscored": {
"type": [
"object",
"null"
],
"description": "Why the scorer did not compare this clip; unreadable when it could not read the artifact. Null when the clip has a pair or no quality attachment exists.",
"properties": {
"reason": {
"type": "string",
"enum": [
"invalid_candidate",
"checksum_mismatch",
"no_reference",
"settings_mismatch",
"unreadable"
]
},
"fields": {
"type": "array",
"items": {
"type": "string"
},
"description": "Settings that differ from the reference."
}
},
"required": [
"reason",
"fields"
]
}
},
"required": [
"hardware",
"pointId",
"slotId",
"caseId",
"mediaPath",
"quality",
"unscored"
]
}
}
},
"required": [
"key",
"scope",
"prompt",
"seed",
"take",
"clips"
]
}
}
},
"required": [
"status",
"groups"
]
},
"options": {
"type": "object",
"additionalProperties": true
},
"state": {
"type": "object",
"additionalProperties": true
},
"workloadKey": {
"type": [
"string",
"null"
]
},
"runs": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": true
},
"description": "Official runs of the selected workload in measurement order, each { id, date } with the GitHub Actions run ID and execution date.",
"x-description-zh": "所选工作负载的正式运行，按测量时间排序；每项为 { id, date }，即 GitHub Actions 运行 ID 和执行日期。"
},
"run": {
"type": [
"object",
"null"
],
"description": "The run the official cells are shown as of (r_date and r_run); null when no official run matches.",
"x-description-zh": "正式结果所截至的运行（由 r_date 和 r_run 决定）；没有匹配的正式运行时为 null。"
},
"chipConfigs": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": true
},
"description": "Every official deployment of the selected workload as { key, point }, the r_configs choices.",
"x-description-zh": "所选工作负载的全部正式部署，每项为 { key, point }，即 r_configs 的可选项。"
},
"history": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": true
},
"description": "Cells for every official run of the r_configs chip configs within r_from and r_to, each run contributing its newest publication of each deployment, plus the named overlay run's cells for those configs; empty when r_configs is empty.",
"x-description-zh": "r_configs 所选芯片配置在 r_from 至 r_to 期间每次正式运行的结果（每次运行中的每个部署取最新发布的结果），以及指定叠加运行中这些配置的结果；r_configs 为空时为空数组。"
},
"excludedCount": {
"type": "integer",
"minimum": 0
},
"sampleFloor": {
"type": "integer",
"const": 10
},
"scheduledClips": {
"type": "integer",
"const": 20
},
"roster": {
"type": "array",
"items": {
"type": "string"
},
"description": "Default GB200, B200, MI355X and H200 slots, plus H100 when qualified approved or exact named diagnostic evidence is available.",
"x-description-zh": "默认包含 GB200、B200、MI355X 和 H200；存在符合门槛的已批准 H100 结果或明确指定的诊断运行时增加 H100。"
},
"metricDefinitions": {
"type": "object",
"additionalProperties": true
},
"plot": {
"type": "object",
"additionalProperties": true
},
"rows": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": true
}
},
"kpis": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": true
}
},
"comparison": {
"type": "object",
"additionalProperties": true
},
"provenance": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": true
}
},
"workload": {
"type": [
"string",
"null"
]
},
"otherWorkloads": {
"type": "integer",
"minimum": 0
},
"coverage": {
"type": "object",
"properties": {
"pagesRead": {
"type": "integer",
"minimum": 1,
"maximum": 5
},
"maxPages": {
"type": "integer",
"enum": [
5
]
},
"nextPage": {
"type": [
"integer",
"null"
]
},
"truncated": {
"type": "boolean"
}
},
"required": [
"pagesRead",
"maxPages",
"nextPage",
"truncated"
]
},
"sources": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": true
}
},
"points": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": true
}
},
"curves": {
"type": "object",
"additionalProperties": true
},
"evidence": {
"type": [
"object",
"null"
],
"additionalProperties": true
},
"discovery": {
"type": "object",
"additionalProperties": true
}
},
"required": [
"view",
"apiVersion",
"params"
],
"additionalProperties": true
}Scroll to inspect the full example
Response example
{
"view": "video",
"apiVersion": "v1",
"params": {}
}Alternate representation: text/csv
Response shape
{
"type": "string"
}Response example
channel,approved,published_at,run_id,artifact_id,source_manifest_sha256,point_id,cell_id,workload_key,workload_sha256,model,hardware,hardware_name,engine,mode,resolution,duration_s,fps,steps,observed_at,participating_gpus,allocated_gpus,scheduled_clips,completed_clips,valid_clips,failed_clips,attempted_clips,warmup_s,client_p50_s,p95_s,clips_per_s,video_seconds_per_s,deadline_s,deadline_attainment,deadline_goodput_clips_per_s,gpu_board_energy_j_per_clip,gpu_board_mean_w,power_valid,power_samples,cost_tier,participating_cost_usd_per_video,participating_cost_usd_per_video_s,participating_cost_usd_per_hour,allocated_cost_usd_per_video,allocated_cost_usd_per_video_s,allocated_cost_usd_per_hour,participating_profit_usd_per_hour,allocated_profit_usd_per_hour
204
No published video artifact is available. The response has no body.
Response shape
{
"type": "null"
}Response example
null404
Source status is preserved; internal details are removed and failures are not cached.
Media type: application/jsonResponse shape
{
"type": "object",
"additionalProperties": true
}Response example
{
"error": "Source data unavailable"
}429
Source status is preserved; internal details are removed and failures are not cached.
Media type: application/jsonResponse shape
{
"type": "object",
"additionalProperties": true
}Response example
{
"error": "Source data unavailable"
}503
Source status is preserved; internal details are removed and failures are not cached.
Media type: application/jsonResponse shape
{
"type": "object",
"additionalProperties": true
}Response example
{
"error": "Source data unavailable"
}400
Invalid or unsupported query.
Media type: application/jsonResponse shape
{
"type": "object",
"additionalProperties": true
}Response example
{
"error": "Invalid parameter",
"param": "model"
}500
Data retrieval or projection failed.
Media type: application/jsonResponse shape
{
"type": "object",
"additionalProperties": true
}Response example
{
"error": "Internal server error"
}GETBeta/api/v1/views/ubenchxuBenchX microbenchmarks with per-device protocols and bandwidth references (mem-bw, sm-l2-distance, tpc-skyline, and core-memory-profiles)
Read-only ubenchx microbenchmarks with per-device protocols and bandwidth references (mem-bw, sm-l2-distance, tpc-skyline, and core-memory-profiles) using the dashboard's source handlers and calculation helpers. Unknown and repeated query keys return 400. The response includes resolved params and preserves missing evidence. Renderer-only styling is not an API parameter. tpc-skyline also returns skuConfigurations separately from measured NVIDIA gpus. gpu=all (default) or Huawei 950PR includes all nine supplied 950PR configurations; an exact Ascend 950PR_<suffix> selects one. Rows preserve Cube/Vector/AICPU counts, MHz clocks, GiB memory, MiB L2, virtualisationOptions and raw ddrRate values with unit=null. Source metadata links the supplied screenshot and marks vendorSourceVerified=false; these are not topology measurements. test=all includes the same array inside tpcSkyline, honoring explicit gpu filters. core-memory-profiles defaults to gpu=all and returns Ascend 950PR_9579 first, followed by B200 SXM; gpu selects either device. B200 retains orderedSms, smInfo, gpcBounds, matrix, stats and metadata, with unit=clock64 cycles and its own colorLimits. The legacy sm-l2-distance API remains available with its original shape. Ascend returns panels with actual coreIds, dieGroups, full matrices, raw SYS_CNT tick units and unique-pair within/between-group statistics; a missing pair class is null. Shared colorLimits and limitations preserve the interpretation. These are profile differences, not core-to-core latency or a floorplan. mem-bw includes Huawei rotating-pool FP16 graph results. timeMs is amortized per copy for metadata.protocol=rotating-pool-npu-graph. peakBandwidthSource documents the per-run normalization reference; optional rows retain observations, bandwidthP10Gbps, bandwidthP90Gbps, processBandwidthGbps, slots and combinedPoolBytes. Missing sizes are listed in metadata.missingSizesBytes. Huawei now covers 8 B through 16 GiB. metadata.smallMessageExtension identifies the separate CANN 9.2.1 run for 8 B–2 KiB, its 4096-byte slot stride, archive digest and cache qualification; the original run metadata applies from 4 KiB upward.
Parameters
| Name | Location | Type | Requirement | Description | Example |
|---|---|---|---|---|---|
| gpu | query | string | Optional | Accelerator key, e.g. "H100 SXM", "Huawei 950PR", "B200 SXM" or "Ascend 950PR_9579". core-memory-profiles and tpc-skyline default to "all"; other tests default to the first available device. "all" returns all devices; unavailable tests retain an empty gpus array for an explicitly selected device. | |
| test | query | string | Optional | uBenchX test selector: mem-bw, sm-l2-distance, tpc-skyline, core-memory-profiles, or all (default). Determines which benchmark data is returned. |
Request
curl
curl -sS 'https://inferencex.semianalysis.com/api/v1/views/ubenchx'Responses
UbenchxView200
Resolved selection and public view data.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"view": {
"type": "string",
"enum": [
"ubenchx"
]
},
"apiVersion": {
"type": "string",
"enum": [
"v1"
]
},
"params": {
"type": "object",
"additionalProperties": true
},
"gpus": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": true
}
},
"tests": {
"type": "array",
"items": {
"type": "string"
}
},
"memBw": {
"type": "object",
"additionalProperties": true
},
"smL2Distance": {
"type": "object",
"additionalProperties": true
},
"tpcSkyline": {
"type": "object",
"additionalProperties": true
},
"skuConfigurations": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": true
}
},
"coreMemoryProfiles": {
"type": "object",
"additionalProperties": true
}
},
"required": [
"view",
"apiVersion",
"params"
],
"additionalProperties": true
}Scroll to inspect the full example
Response example
{
"view": "ubenchx",
"apiVersion": "v1",
"params": {}
}404
Source status is preserved; internal details are removed and failures are not cached.
Media type: application/jsonResponse shape
{
"type": "object",
"additionalProperties": true
}Response example
{
"error": "Source data unavailable"
}429
Source status is preserved; internal details are removed and failures are not cached.
Media type: application/jsonResponse shape
{
"type": "object",
"additionalProperties": true
}Response example
{
"error": "Source data unavailable"
}503
Source status is preserved; internal details are removed and failures are not cached.
Media type: application/jsonResponse shape
{
"type": "object",
"additionalProperties": true
}Response example
{
"error": "Source data unavailable"
}400
Invalid or unsupported query.
Media type: application/jsonResponse shape
{
"type": "object",
"additionalProperties": true
}Response example
{
"error": "Invalid parameter",
"param": "model"
}500
Data retrieval or projection failed.
Media type: application/jsonResponse shape
{
"type": "object",
"additionalProperties": true
}Response example
{
"error": "Internal server error"
}GETBeta/api/v1/views/model-architecturesRead model architecture specifications
Returns the same compiled architecture registry used by the English and Chinese model pages and dashboard diagrams. Omit model for all registry entries, including models without a canonical compare slug (slug is null). Supply an exact model display name or canonical compare slug to filter entries; known models without architecture metadata return an empty models array; aliases, empty values, unknown or repeated parameters return 400. Optional facts remain omitted when unspecified. Parameter counts use billions, dimensions and context windows use exact integer counts; sourceUrl identifies the vendor source. Diagram expansion is presentation only. This catalog reads no database, benchmark measurements, private data or unofficial runs; use the inference view for model-page benchmark controls. Public, no authentication, JSON only. Successful responses use Cache-Control: public, max-age=0, s-maxage=300.
Parameters
| Name | Location | Type | Requirement | Description | Example |
|---|---|---|---|---|---|
| model | query | string | Optional | Exact display name or canonical compare slug. Omit to return all architectures; empty values and aliases are rejected. | kimi-k3 |
Request
curl
curl -sS 'https://inferencex.semianalysis.com/api/v1/views/model-architectures?model=kimi-k3'Responses
ModelArchitecturesView200
Architecture facts with resolved model selection and canonical slugs.
Response shape
{
"type": "object",
"properties": {
"view": {
"type": "string",
"enum": [
"model-architectures"
]
},
"apiVersion": {
"type": "string",
"enum": [
"v1"
]
},
"params": {
"type": "object",
"properties": {
"model": {
"type": [
"string",
"null"
]
}
},
"required": [
"model"
],
"additionalProperties": false
},
"models": {
"type": "array",
"items": {
"type": "object",
"properties": {
"slug": {
"type": [
"string",
"null"
]
},
"architecture": {
"type": "object",
"properties": {
"model": {
"type": "string"
},
"totalParams": {
"type": "number",
"description": "Billions of parameters; see model-specific features for excluded auxiliary modules."
},
"activeParams": {
"type": "number",
"description": "Billions of active parameters per token; decode count when activeParamsPrefill is present."
},
"activeParamsPrefill": {
"type": "number"
},
"conditionalMemoryParams": {
"type": "number"
},
"conditionalMemoryName": {
"type": "string"
},
"architectureType": {
"type": "string",
"enum": [
"dense",
"moe"
]
},
"attentionType": {
"type": "string"
},
"attentionLabel": {
"type": "string"
},
"numLayers": {
"type": "integer"
},
"hiddenSize": {
"type": "integer"
},
"numHeads": {
"type": "integer"
},
"numKVHeads": {
"type": "integer"
},
"headDim": {
"type": "integer"
},
"vocabSize": {
"type": "integer"
},
"ffnDim": {
"type": "integer"
},
"numExperts": {
"type": "integer"
},
"activeExperts": {
"type": "integer"
},
"hasSharedExpert": {
"type": "boolean"
},
"sharedExperts": {
"type": "integer"
},
"denseFFNLayers": {
"type": "integer"
},
"denseFFNDim": {
"type": "integer"
},
"hashRoutedLayers": {
"type": "integer"
},
"slidingWindow": {
"type": "integer"
},
"hyperConnections": {
"type": "integer"
},
"contextWindow": {
"type": "integer"
},
"advertisedContextWindow": {
"type": "integer"
},
"features": {
"type": "array",
"items": {
"type": "string"
}
},
"featuresZh": {
"type": "array",
"items": {
"type": "string"
}
},
"developer": {
"type": "string"
},
"sourceUrl": {
"type": "string"
},
"attentionExpandable": {
"type": "boolean"
},
"alternatingAttentionExpandable": {
"type": "boolean"
},
"alternatingNote": {
"type": "string"
},
"alternatingNoteZh": {
"type": "string"
},
"denseLayerAttentionLabel": {
"type": "string"
},
"ffnVariant": {
"type": "string"
},
"ffnGateActivation": {
"type": "string"
},
"ffnSwiGLUOAI": {
"type": "boolean"
},
"ffnClamp": {
"type": "number"
},
"routedExpertHiddenSize": {
"type": "integer"
},
"residualType": {
"type": "string",
"enum": [
"AttnRes",
"gated"
]
},
"alternatingLayers": {
"type": "array",
"items": {
"type": "object",
"properties": {
"label": {
"type": "string"
},
"description": {
"type": "string"
},
"count": {
"type": "integer"
},
"colorKey": {
"type": "string"
},
"slidingWindow": {
"type": "integer"
},
"attentionShape": {
"type": "string"
}
},
"required": [
"label",
"description",
"count",
"colorKey"
],
"additionalProperties": false
}
},
"encoderDecoder": {
"type": "object",
"properties": {
"encoderLayers": {
"type": "integer"
},
"decoderLayers": {
"type": "integer"
},
"globalKVSource": {
"type": "string",
"enum": [
"final-encoder-hidden-states"
]
},
"encoderSlidingOnlyLayers": {
"type": "integer"
},
"decoderAttention": {
"type": "string",
"enum": [
"joint-local-swa-and-shared-global-kv"
]
}
},
"required": [
"encoderLayers",
"decoderLayers",
"globalKVSource",
"encoderSlidingOnlyLayers",
"decoderAttention"
],
"additionalProperties": false
}
},
"required": [
"model",
"totalParams",
"activeParams",
"architectureType",
"attentionType"
],
"additionalProperties": true
}
},
"required": [
"slug",
"architecture"
],
"additionalProperties": false
}
}
},
"required": [
"view",
"apiVersion",
"params",
"models"
],
"additionalProperties": false
}Scroll to inspect the full example
Response example
{
"view": "model-architectures",
"apiVersion": "v1",
"params": {
"model": "Kimi-K3"
},
"models": [
{
"slug": "kimi-k3",
"architecture": {
"model": "Kimi-K3",
"totalParams": 2800,
"activeParams": 104,
"architectureType": "moe",
"attentionType": "Hybrid",
"numLayers": 93,
"hiddenSize": 7168,
"numExperts": 898,
"activeExperts": 16,
"sourceUrl": "https://huggingface.co/moonshotai/Kimi-K3"
}
}
]
}Scroll to inspect the full example
400
Invalid model, empty model, unknown parameter or repeated parameter.
Response shape
{
"type": "object",
"properties": {
"error": {
"type": "string"
},
"param": {
"type": "string"
}
},
"required": [
"error"
],
"additionalProperties": true
}Scroll to inspect the full example
Response example
{
"error": "Unknown model: missing",
"param": "model"
}500
Architecture catalog assembly failed.
Response shape
{
"type": "object",
"properties": {
"error": {
"type": "string"
}
},
"required": [
"error"
],
"additionalProperties": true
}Response example
{
"error": "Internal server error"
}Core benchmark data
Benchmark results, availability, workflow provenance, evaluations, and reliability.
GETStable/api/v1/availabilityList available benchmark configurations
Returns model, sequence, precision, hardware, framework, speculative method, benchmark type, and date combinations that have benchmark data.
Parameters
No parameters
Request
curl
curl -sS 'https://inferencex.semianalysis.com/api/v1/availability'Responses
AvailabilityRows200
Available configuration rows.
Media type: application/jsonResponse shape
{
"type": "array",
"items": {
"type": "object",
"properties": {
"model": {
"type": "string"
},
"isl": {
"type": [
"number",
"null"
]
},
"osl": {
"type": [
"number",
"null"
]
},
"precision": {
"type": "string"
},
"hardware": {
"type": "string"
},
"framework": {
"type": "string"
},
"spec_method": {
"type": "string"
},
"disagg": {
"type": "boolean"
},
"benchmark_type": {
"type": "string"
},
"date": {
"type": "string",
"format": "date"
}
},
"required": [
"model",
"isl",
"osl",
"precision",
"hardware",
"framework",
"spec_method",
"disagg",
"benchmark_type",
"date"
],
"additionalProperties": false
}
}Scroll to inspect the full example
Response example
[
{
"model": "dsr1",
"isl": 1024,
"osl": 1024,
"precision": "fp8",
"hardware": "h200_sxm",
"framework": "vllm",
"spec_method": "none",
"disagg": false,
"benchmark_type": "single_turn",
"date": "2026-08-08"
}
]Scroll to inspect the full example
500
The availability query failed.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"error": {
"type": "string"
}
},
"required": [
"error"
],
"additionalProperties": true
}Response example
{
"error": "Internal server error"
}GETStable/api/v1/benchmarksRead benchmark results
Returns raw benchmark rows for a display model. Use date for an as-of snapshot, exact=true for that exact date, runId to constrain the latest lookup, or exactRun=true with a numeric runId for that run’s logical snapshot. An AgentX curve is scoped by model, hardware, framework, precision, and workload: a normal run replaces all prior topology, speculative-decoding, and offload variants in that curve. Explicit append-only runs may include the preceding same-image snapshot; producer IDs and URLs are preserved. TPU tooltips and point-detail headers omit GitHub run links; API run_url remains the original producer URL. Official preview results published outside CI carry their source page as run_url, or null. Historical snapshots remain accessible. view=calculator returns a trimmed page-owned projection (measured power metrics and workers are removed; its allowlist may change). powerValid=strictV2 selects validated schema-v2 power measurements and cannot be combined with view=calculator. Omit powerValid to keep general benchmark results regardless of power validity.
Parameters
| Name | Location | Type | Requirement | Description | Example |
|---|---|---|---|---|---|
| model | query | string | Required | Display model name. GLM-5.3-Flash maps only to glm5.3flash, separately from GLM-5.2/5.3, and supports 8k/1k and agentic-traces. Registration does not imply published observations or hardware qualification. Unofficial-run overlays are separate from published benchmark results. The web UI shows GLM-5.3-Flash only when its feature gate is enabled. This controls presentation, not access; read-only API model parameters and returned data are unaffected. | DeepSeek-R1-0528 |
| date | query | date | Optional | Latest data on or before this calendar date (YYYY-MM-DD), unless exact is true. An empty value means the latest data; any other value, such as latest, yields 400. | 2026-08-08 |
| exact | query | boolean | Optional | Set true to require the supplied date exactly. Only true and false are accepted. | false |
| runId | query | integer | Optional | Positive integer GitHub Actions run ID. Other values yield 400. | 123456789 |
| exactRun | query | boolean | Optional | With a numeric runId, return that run’s logical snapshot. Append-only runs include predecessors whose shared topologies keep the same image. Only true and false are accepted, and true requires runId. | false |
| view | query | enum | Optional | calculator trims each row to the page-owned metric allowlist the throughput calculator consumes and removes workers; measured power metrics are excluded from this view. Requires sequence. Omit for every stored metric, including measured power. Other values yield 400 Unknown view. | calculator |
| sequence | query | enum | Optional | Required when view=calculator and ignored otherwise. Unknown values yield 400 Unknown calculator sequence. | 1k/1k |
| powerValid | query | enum | Optional | Only strictV2 is accepted. It keeps rows whose metrics.power_valid is the number 1 and metrics.power_metric_schema_version is the number 2 (whole-deployment energy semantics). Omit this parameter to apply no power filter, preserving throughput and latency results even when power is missing or invalid. Other values, including an empty value, yield 400 Unknown powerValid filter. Cannot be combined with view=calculator. | strictV2 |
Request
curl
curl -sS 'https://inferencex.semianalysis.com/api/v1/benchmarks?model=DeepSeek-R1-0528'Responses
BenchmarkRows200
Benchmark rows with scalar metrics in the metrics object.
Media type: application/jsonResponse shape
{
"type": "array",
"items": {
"type": "object",
"properties": {
"id": {
"type": "integer"
},
"hardware": {
"type": "string"
},
"framework": {
"type": "string"
},
"model": {
"type": "string"
},
"precision": {
"type": "string"
},
"spec_method": {
"type": "string"
},
"disagg": {
"type": "boolean"
},
"is_multinode": {
"type": "boolean"
},
"prefill_tp": {
"type": "integer"
},
"prefill_ep": {
"type": "integer"
},
"prefill_dp_attention": {
"type": "boolean"
},
"prefill_num_workers": {
"type": "integer"
},
"decode_tp": {
"type": "integer"
},
"decode_ep": {
"type": "integer"
},
"decode_dp_attention": {
"type": "boolean"
},
"decode_num_workers": {
"type": "integer"
},
"num_prefill_gpu": {
"type": "integer",
"description": "Physical prefill chips; aggregate engines may mirror their single chip count in both role columns."
},
"num_decode_gpu": {
"type": "integer",
"description": "Physical decode chips, independent of logical TP and DP. Sum role counts only for disaggregated engines."
},
"benchmark_type": {
"type": "string"
},
"isl": {
"type": [
"number",
"null"
]
},
"osl": {
"type": [
"number",
"null"
]
},
"conc": {
"type": "integer"
},
"offload_mode": {
"type": "string"
},
"image": {
"type": [
"string",
"null"
]
},
"recipe_fingerprint": {
"type": [
"string",
"null"
]
},
"metrics": {
"type": "object",
"additionalProperties": {
"type": "number"
},
"description": "Scalar metric map. Time metrics, including p99_itl and p99_tpot, are in seconds. p99_itl measures inter-token latency; p99_tpot measures per-request time per output token. Use the actual p99_itl field for an inter-token latency requirement, not the reciprocal of p99_intvty. Throughput metrics use tokens per second per GPU unless their name states otherwise; output_tput_per_gpu counts output tokens. On disaggregated rows tput_per_gpu divides by every chip, input_tput_per_gpu by prefill chips and output_tput_per_gpu by decode chips, for fixed-sequence and agentic runs alike. Agentic disaggregated rows also keep the original per-total-chip rates from the producer as input_tput_per_total_gpu and output_tput_per_total_gpu. Keys evolve independently; measured power / energy / GPU-telemetry keys are typed below.",
"properties": {
"power_valid": {
"type": "number",
"description": "Publication verdict: 1 = validated measurement window; 0 = failed validation — measured power/energy values are withheld from this row end-to-end, so treat any that remain as unreliable; absent = no validation verdict is available in this response. Absence alone establishes neither the reason, the measurement age, nor invalidity."
},
"power_metric_schema_version": {
"type": "number",
"description": "Power schema version. Version 2 defines every unprefixed joules_per_* field as whole-deployment energy, including on disaggregated runs."
},
"cpu_power_valid": {
"type": "number",
"description": "CPU-side leg verdict (NVL72 Grace sockets or single-node AMD EPYC RAPL packages), independent of power_valid: 1 = the CPU-side (and module, when present) keys were integrated over the validated window with full socket coverage; 0 = the leg failed and no CPU-side keys are emitted; absent = the run had no CPU-side leg. Ingest withholds the CPU-side keys unless this verdict is 1, and power_valid = 0 does not remove them. GPU power and energy never change because of this verdict."
},
"avg_power_w": {
"type": "number",
"description": "Mean per-GPU power draw in watts during the measured load window."
},
"avg_total_gpu_power_w": {
"type": "number",
"description": "Mean total GPU power draw in watts across the measured deployment."
},
"total_gpu_energy_j": {
"type": "number",
"description": "Total GPU energy integrated over the measured deployment load window, in joules."
},
"p75_power_w": {
"type": "number",
"description": "Time-weighted P75 of synchronized total GPU-board watts, divided by participating GPU count, over the validated benchmark window with piecewise-linear interpolation. Not a percentile of individual-device percentiles; absent when not measured."
},
"p75_total_gpu_power_w": {
"type": "number",
"description": "Time-weighted P75 of synchronized total GPU-board watts over the validated benchmark window with piecewise-linear interpolation. Accelerator boards only, not chassis or facility power."
},
"p90_power_w": {
"type": "number",
"description": "Time-weighted P90 of synchronized total GPU-board watts, divided by participating GPU count, over the validated benchmark window with piecewise-linear interpolation. Not a percentile of individual-device percentiles; absent when not measured."
},
"p90_total_gpu_power_w": {
"type": "number",
"description": "Time-weighted P90 of synchronized total GPU-board watts over the validated benchmark window with piecewise-linear interpolation. Accelerator boards only, not chassis or facility power."
},
"joules_per_successful_query": {
"type": "number",
"description": "Whole-deployment energy in joules divided by successful requests."
},
"joules_per_output_token": {
"type": "number",
"description": "Energy per generated output token in joules; cluster-wide on schema-version-2 rows, including disaggregated runs."
},
"joules_per_total_token": {
"type": "number",
"description": "Total system energy divided by input plus output tokens; a workload-shape-fair view that does not treat prompt tokens as free."
},
"prefill_avg_power_w": {
"type": "number",
"description": "Mean per-GPU power draw in watts across prefill workers; emitted only for deployments with distinct prefill and decode roles."
},
"decode_avg_power_w": {
"type": "number",
"description": "Mean per-GPU power draw in watts across decode workers; emitted only for deployments with distinct prefill and decode roles."
},
"joules_per_input_token": {
"type": "number",
"description": "Energy per input token in joules; cluster-wide on schema-version-2 rows."
},
"prefill_joules_per_input_token": {
"type": "number",
"description": "Role-local prefill energy per input token in joules."
},
"decode_joules_per_output_token": {
"type": "number",
"description": "Role-local decode energy per generated output token in joules."
},
"avg_temp_c": {
"type": "number",
"description": "Mean per-GPU temperature in degrees Celsius during the load window."
},
"peak_temp_c": {
"type": "number",
"description": "Maximum instantaneous per-GPU temperature in degrees Celsius during the load window."
},
"avg_util_pct": {
"type": "number",
"description": "Mean per-GPU utilization percentage (0-100) during the load window."
},
"avg_mem_used_mb": {
"type": "number",
"description": "Mean per-GPU memory used in MB during the load window."
},
"avg_cpu_socket_power_w": {
"type": "number",
"description": "Mean over Grace sockets of each socket’s window-mean Grace-side power in watts (ACPI hwmon Grace Power Socket sensor, which NVIDIA defines only as the Grace socket power; modeled as the CPU and SysIO rails, LPDDR5X and socket regulation) over the same validated window as GPU energy. On single-node AMD rows (power_audit.cpu.sensor_kind = rapl_package) it is instead each EPYC package’s RAPL energy-counter power, which has no DRAM domain."
},
"avg_total_cpu_power_w": {
"type": "number",
"description": "Sum over every CPU socket in the deployment of window-mean socket power in watts (Grace side on NVL72, RAPL package on single-node AMD); dividing by avg_cpu_socket_power_w recovers the socket count (two per compute tray or AMD host)."
},
"total_cpu_energy_j": {
"type": "number",
"description": "CPU-side energy in joules (Grace side on NVL72, RAPL package on single-node AMD) integrated over the validated window across every socket. Additive to total_gpu_energy_j, never included in it."
},
"avg_total_module_power_w": {
"type": "number",
"description": "Sum over sockets of window-mean compute-module power in watts (Grace + its Blackwell GPUs + HBM + LPDDR5X + regulator loss, ACPI hwmon Module Power Socket sensor); emitted only when the module sensor exists on every socket. It already contains the GPU-board watts, so do not add avg_total_gpu_power_w to it."
},
"total_module_energy_j": {
"type": "number",
"description": "Compute-module energy in joules over the validated window across every socket; emitted with avg_total_module_power_w and includes the GPU energy."
},
"avg_total_cpu_rail_power_w": {
"type": "number",
"description": "Sum over Grace sockets of window-mean CPU core rail power in watts (ACPI CPU Power Socket sensor, no SysIO); emitted only when every socket reports a gap-free rail. A breakdown inside avg_total_cpu_power_w, not additive to it. NVL72 only."
},
"total_cpu_rail_energy_j": {
"type": "number",
"description": "CPU core rail energy in joules over the validated window across every socket; emitted with avg_total_cpu_rail_power_w and already included in total_cpu_energy_j."
},
"avg_total_cpu_sysio_power_w": {
"type": "number",
"description": "Sum over Grace sockets of window-mean SysIO rail power in watts (ACPI SysIO Power Socket sensor); emitted only when every socket reports a gap-free rail. A breakdown inside avg_total_cpu_power_w, not additive to it. NVL72 only."
},
"total_cpu_sysio_energy_j": {
"type": "number",
"description": "SysIO rail energy in joules over the validated window across every socket; emitted with avg_total_cpu_sysio_power_w and already included in total_cpu_energy_j."
}
}
},
"workers": {
"type": "array",
"items": {
"type": "object",
"properties": {
"role": {
"type": "string"
},
"worker_idx": {
"type": "integer"
},
"hosts": {
"type": "array",
"items": {
"type": "string"
}
},
"num_gpus": {
"type": "integer"
},
"avg_power_w": {
"type": "number"
},
"avg_temp_c": {
"type": "number"
},
"peak_temp_c": {
"type": "number"
},
"avg_util_pct": {
"type": "number"
},
"avg_mem_used_mb": {
"type": "number"
}
},
"required": [
"role",
"worker_idx",
"num_gpus",
"avg_power_w"
],
"additionalProperties": false
}
},
"power_invalid_reasons": {
"type": "array",
"items": {
"type": "string"
},
"description": "Producer snake_case reason codes explaining a withheld measurement, present when metrics.power_valid == 0. Absent on legacy rows and validated rows."
},
"power_audit": {
"type": "object",
"properties": {
"window_start_unix": {
"type": "number"
},
"window_end_unix": {
"type": "number"
},
"expected_gpu_count": {
"type": "integer"
},
"observed_gpu_count": {
"type": "integer"
},
"sample_count": {
"type": "integer"
},
"max_sample_gap_s": {
"type": "number"
},
"producer_sha": {
"type": [
"string",
"null"
]
},
"exporter_image_sha256": {
"type": [
"string",
"null"
]
},
"source": {
"type": "string"
},
"observed_gpu_ids": {
"type": "array",
"items": {
"type": "string"
}
},
"cpu": {
"type": "object",
"properties": {
"sensor_kind": {
"type": "string",
"enum": [
"module",
"grace_socket",
"dcgm_cpu_rail",
"rapl_package"
]
},
"source": {
"type": "string",
"description": "Collector that produced the CPU-side samples: acpi (hwmon), dcgm, or rapl (AMD EPYC package energy counter)."
},
"expected_sockets": {
"type": "integer"
},
"observed_sockets": {
"type": "integer"
},
"sample_row_count": {
"type": "integer"
},
"reason_codes": {
"type": "array",
"items": {
"type": "string"
}
}
},
"additionalProperties": true,
"description": "CPU-side leg provenance emitted alongside cpu_power_valid: the sensor kind that fed the CPU-side keys (on NVL72, module preferred over grace_socket over dcgm_cpu_rail; rapl_package on single-node AMD EPYC hosts), its collector, expected vs. observed sockets (two per compute tray or AMD host), stored sample rows, and the leg’s reason codes (empty when valid). Absent on runs without the CPU-side leg."
}
},
"additionalProperties": true,
"description": "Compact power measurement-window audit emitted alongside the power_valid verdict: window bounds, expected vs. observed GPU counts, sample statistics, and producer identity (producer_sha / exporter_image_sha256 are null for single-node telemetry without an srt-slurm producer). source names the validation document: a relative path within the source run artifact bundle or, on AgentX rows whose retained window ingest or backfill recovered, the alias power_validation_<stem>_conc<N>.json for LOGS/agentic/conc_<N>/power_validation.json in the power_audit_<stem> artifact (such rows carry only source and the window bounds); observed_gpu_ids contains producer device identifiers, which may be indices rather than physical UUIDs. cpu carries the CPU-side leg’s provenance (NVL72 or single-node AMD) when that leg ran. Present on valid and invalid rows when emitted; absence alone does not establish age or validity."
},
"date": {
"type": "string",
"format": "date"
},
"workflow_run_id": {
"type": "integer"
},
"run_started_at": {
"type": [
"string",
"null"
],
"format": "date-time"
},
"run_url": {
"type": [
"string",
"null"
]
}
},
"required": [
"id",
"hardware",
"framework",
"model",
"precision",
"spec_method",
"disagg",
"is_multinode",
"prefill_tp",
"prefill_ep",
"prefill_dp_attention",
"prefill_num_workers",
"decode_tp",
"decode_ep",
"decode_dp_attention",
"decode_num_workers",
"num_prefill_gpu",
"num_decode_gpu",
"benchmark_type",
"isl",
"osl",
"conc",
"offload_mode",
"image",
"recipe_fingerprint",
"metrics",
"date",
"run_url"
],
"additionalProperties": false
}
}Scroll to inspect the full example
Response example
[
{
"id": 421,
"hardware": "h200_sxm",
"framework": "vllm",
"model": "dsr1",
"precision": "fp8",
"spec_method": "none",
"disagg": false,
"is_multinode": false,
"prefill_tp": 8,
"prefill_ep": 1,
"prefill_dp_attention": false,
"prefill_num_workers": 1,
"decode_tp": 8,
"decode_ep": 1,
"decode_dp_attention": false,
"decode_num_workers": 1,
"num_prefill_gpu": 0,
"num_decode_gpu": 8,
"benchmark_type": "single_turn",
"isl": 1024,
"osl": 1024,
"conc": 32,
"offload_mode": "off",
"image": "vllm/vllm-openai:v0.10.2",
"recipe_fingerprint": "7d72a33d7d72a33d7d72a33d7d72a33d7d72a33d7d72a33d7d72a33d7d72a33d",
"metrics": {
"median_ttft": 0.42,
"median_tpot": 0.018,
"tput_per_gpu": 128.4,
"power_valid": 1,
"power_metric_schema_version": 2,
"avg_power_w": 678.5,
"joules_per_output_token": 5.3,
"joules_per_total_token": 2.65,
"avg_temp_c": 61.2
},
"date": "2026-08-08",
"run_url": "https://github.com/semianalysis/inference-benchmarks/actions/runs/123456789"
}
]Scroll to inspect the full example
400
A query parameter is malformed, so nothing is read from the database: the model is missing or unsupported, date is not a YYYY-MM-DD calendar date, exact or exactRun is not true or false, runId is not a positive integer, exactRun=true has no runId, view is not calculator, the calculator sequence is unknown, a supplied powerValid value is not strictV2, or powerValid=strictV2 is combined with view=calculator. Unknown parameters are ignored.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"error": {
"type": "string"
}
},
"required": [
"error"
],
"additionalProperties": true
}Response example
{
"error": "Unknown model"
}500
The benchmark query failed.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"error": {
"type": "string"
}
},
"required": [
"error"
],
"additionalProperties": true
}Response example
{
"error": "Internal server error"
}GETStable/api/v1/benchmarks/historyRead benchmark history
Returns historical logical snapshots for one model and either a fixed input/output token pair or Agentic Traces. Replaced AgentX variants remain in their earlier snapshots. Append-only snapshots include inherited points with original producer metadata and separate curve snapshot metadata.
Parameters
| Name | Location | Type | Requirement | Description | Example |
|---|---|---|---|---|---|
| model | query | string | Required | Display model name. | DeepSeek-R1-0528 |
| isl | query | integer | Optional | Positive input sequence length in tokens. Required unless benchmarkType=agentic_traces. | 1024 |
| osl | query | integer | Optional | Positive output sequence length in tokens. Required unless benchmarkType=agentic_traces. | 1024 |
| benchmarkType | query | string | Optional | Set to agentic_traces to read Agentic Traces history without ISL/OSL. | agentic_traces |
| view | query | enum | Optional | calculator trims each row to the metrics the throughput calculator consumes, for a smaller payload. Omit to retain other history metrics, including measured power when present. Both views omit mean_* and std_* statistics for ttft, tpot, itl, e2el, and intvty. | calculator |
Request
curl
curl -sS 'https://inferencex.semianalysis.com/api/v1/benchmarks/history?model=DeepSeek-R1-0528&isl=1024&osl=1024'Responses
BenchmarkRows200
Historical benchmark rows.
Media type: application/jsonResponse shape
{
"type": "array",
"items": {
"type": "object",
"properties": {
"id": {
"type": "integer"
},
"hardware": {
"type": "string"
},
"framework": {
"type": "string"
},
"model": {
"type": "string"
},
"precision": {
"type": "string"
},
"spec_method": {
"type": "string"
},
"disagg": {
"type": "boolean"
},
"is_multinode": {
"type": "boolean"
},
"prefill_tp": {
"type": "integer"
},
"prefill_ep": {
"type": "integer"
},
"prefill_dp_attention": {
"type": "boolean"
},
"prefill_num_workers": {
"type": "integer"
},
"decode_tp": {
"type": "integer"
},
"decode_ep": {
"type": "integer"
},
"decode_dp_attention": {
"type": "boolean"
},
"decode_num_workers": {
"type": "integer"
},
"num_prefill_gpu": {
"type": "integer",
"description": "Physical prefill chips; aggregate engines may mirror their single chip count in both role columns."
},
"num_decode_gpu": {
"type": "integer",
"description": "Physical decode chips, independent of logical TP and DP. Sum role counts only for disaggregated engines."
},
"benchmark_type": {
"type": "string"
},
"isl": {
"type": [
"number",
"null"
]
},
"osl": {
"type": [
"number",
"null"
]
},
"conc": {
"type": "integer"
},
"offload_mode": {
"type": "string"
},
"image": {
"type": [
"string",
"null"
]
},
"recipe_fingerprint": {
"type": [
"string",
"null"
]
},
"metrics": {
"type": "object",
"additionalProperties": {
"type": "number"
},
"description": "Scalar metric map. Time metrics, including p99_itl and p99_tpot, are in seconds. p99_itl measures inter-token latency; p99_tpot measures per-request time per output token. Use the actual p99_itl field for an inter-token latency requirement, not the reciprocal of p99_intvty. Throughput metrics use tokens per second per GPU unless their name states otherwise; output_tput_per_gpu counts output tokens. On disaggregated rows tput_per_gpu divides by every chip, input_tput_per_gpu by prefill chips and output_tput_per_gpu by decode chips, for fixed-sequence and agentic runs alike. Agentic disaggregated rows also keep the original per-total-chip rates from the producer as input_tput_per_total_gpu and output_tput_per_total_gpu. Keys evolve independently; measured power / energy / GPU-telemetry keys are typed below.",
"properties": {
"power_valid": {
"type": "number",
"description": "Publication verdict: 1 = validated measurement window; 0 = failed validation — measured power/energy values are withheld from this row end-to-end, so treat any that remain as unreliable; absent = no validation verdict is available in this response. Absence alone establishes neither the reason, the measurement age, nor invalidity."
},
"power_metric_schema_version": {
"type": "number",
"description": "Power schema version. Version 2 defines every unprefixed joules_per_* field as whole-deployment energy, including on disaggregated runs."
},
"cpu_power_valid": {
"type": "number",
"description": "CPU-side leg verdict (NVL72 Grace sockets or single-node AMD EPYC RAPL packages), independent of power_valid: 1 = the CPU-side (and module, when present) keys were integrated over the validated window with full socket coverage; 0 = the leg failed and no CPU-side keys are emitted; absent = the run had no CPU-side leg. Ingest withholds the CPU-side keys unless this verdict is 1, and power_valid = 0 does not remove them. GPU power and energy never change because of this verdict."
},
"avg_power_w": {
"type": "number",
"description": "Mean per-GPU power draw in watts during the measured load window."
},
"avg_total_gpu_power_w": {
"type": "number",
"description": "Mean total GPU power draw in watts across the measured deployment."
},
"total_gpu_energy_j": {
"type": "number",
"description": "Total GPU energy integrated over the measured deployment load window, in joules."
},
"p75_power_w": {
"type": "number",
"description": "Time-weighted P75 of synchronized total GPU-board watts, divided by participating GPU count, over the validated benchmark window with piecewise-linear interpolation. Not a percentile of individual-device percentiles; absent when not measured."
},
"p75_total_gpu_power_w": {
"type": "number",
"description": "Time-weighted P75 of synchronized total GPU-board watts over the validated benchmark window with piecewise-linear interpolation. Accelerator boards only, not chassis or facility power."
},
"p90_power_w": {
"type": "number",
"description": "Time-weighted P90 of synchronized total GPU-board watts, divided by participating GPU count, over the validated benchmark window with piecewise-linear interpolation. Not a percentile of individual-device percentiles; absent when not measured."
},
"p90_total_gpu_power_w": {
"type": "number",
"description": "Time-weighted P90 of synchronized total GPU-board watts over the validated benchmark window with piecewise-linear interpolation. Accelerator boards only, not chassis or facility power."
},
"joules_per_successful_query": {
"type": "number",
"description": "Whole-deployment energy in joules divided by successful requests."
},
"joules_per_output_token": {
"type": "number",
"description": "Energy per generated output token in joules; cluster-wide on schema-version-2 rows, including disaggregated runs."
},
"joules_per_total_token": {
"type": "number",
"description": "Total system energy divided by input plus output tokens; a workload-shape-fair view that does not treat prompt tokens as free."
},
"prefill_avg_power_w": {
"type": "number",
"description": "Mean per-GPU power draw in watts across prefill workers; emitted only for deployments with distinct prefill and decode roles."
},
"decode_avg_power_w": {
"type": "number",
"description": "Mean per-GPU power draw in watts across decode workers; emitted only for deployments with distinct prefill and decode roles."
},
"joules_per_input_token": {
"type": "number",
"description": "Energy per input token in joules; cluster-wide on schema-version-2 rows."
},
"prefill_joules_per_input_token": {
"type": "number",
"description": "Role-local prefill energy per input token in joules."
},
"decode_joules_per_output_token": {
"type": "number",
"description": "Role-local decode energy per generated output token in joules."
},
"avg_temp_c": {
"type": "number",
"description": "Mean per-GPU temperature in degrees Celsius during the load window."
},
"peak_temp_c": {
"type": "number",
"description": "Maximum instantaneous per-GPU temperature in degrees Celsius during the load window."
},
"avg_util_pct": {
"type": "number",
"description": "Mean per-GPU utilization percentage (0-100) during the load window."
},
"avg_mem_used_mb": {
"type": "number",
"description": "Mean per-GPU memory used in MB during the load window."
},
"avg_cpu_socket_power_w": {
"type": "number",
"description": "Mean over Grace sockets of each socket’s window-mean Grace-side power in watts (ACPI hwmon Grace Power Socket sensor, which NVIDIA defines only as the Grace socket power; modeled as the CPU and SysIO rails, LPDDR5X and socket regulation) over the same validated window as GPU energy. On single-node AMD rows (power_audit.cpu.sensor_kind = rapl_package) it is instead each EPYC package’s RAPL energy-counter power, which has no DRAM domain."
},
"avg_total_cpu_power_w": {
"type": "number",
"description": "Sum over every CPU socket in the deployment of window-mean socket power in watts (Grace side on NVL72, RAPL package on single-node AMD); dividing by avg_cpu_socket_power_w recovers the socket count (two per compute tray or AMD host)."
},
"total_cpu_energy_j": {
"type": "number",
"description": "CPU-side energy in joules (Grace side on NVL72, RAPL package on single-node AMD) integrated over the validated window across every socket. Additive to total_gpu_energy_j, never included in it."
},
"avg_total_module_power_w": {
"type": "number",
"description": "Sum over sockets of window-mean compute-module power in watts (Grace + its Blackwell GPUs + HBM + LPDDR5X + regulator loss, ACPI hwmon Module Power Socket sensor); emitted only when the module sensor exists on every socket. It already contains the GPU-board watts, so do not add avg_total_gpu_power_w to it."
},
"total_module_energy_j": {
"type": "number",
"description": "Compute-module energy in joules over the validated window across every socket; emitted with avg_total_module_power_w and includes the GPU energy."
},
"avg_total_cpu_rail_power_w": {
"type": "number",
"description": "Sum over Grace sockets of window-mean CPU core rail power in watts (ACPI CPU Power Socket sensor, no SysIO); emitted only when every socket reports a gap-free rail. A breakdown inside avg_total_cpu_power_w, not additive to it. NVL72 only."
},
"total_cpu_rail_energy_j": {
"type": "number",
"description": "CPU core rail energy in joules over the validated window across every socket; emitted with avg_total_cpu_rail_power_w and already included in total_cpu_energy_j."
},
"avg_total_cpu_sysio_power_w": {
"type": "number",
"description": "Sum over Grace sockets of window-mean SysIO rail power in watts (ACPI SysIO Power Socket sensor); emitted only when every socket reports a gap-free rail. A breakdown inside avg_total_cpu_power_w, not additive to it. NVL72 only."
},
"total_cpu_sysio_energy_j": {
"type": "number",
"description": "SysIO rail energy in joules over the validated window across every socket; emitted with avg_total_cpu_sysio_power_w and already included in total_cpu_energy_j."
}
}
},
"workers": {
"type": "array",
"items": {
"type": "object",
"properties": {
"role": {
"type": "string"
},
"worker_idx": {
"type": "integer"
},
"hosts": {
"type": "array",
"items": {
"type": "string"
}
},
"num_gpus": {
"type": "integer"
},
"avg_power_w": {
"type": "number"
},
"avg_temp_c": {
"type": "number"
},
"peak_temp_c": {
"type": "number"
},
"avg_util_pct": {
"type": "number"
},
"avg_mem_used_mb": {
"type": "number"
}
},
"required": [
"role",
"worker_idx",
"num_gpus",
"avg_power_w"
],
"additionalProperties": false
}
},
"power_invalid_reasons": {
"type": "array",
"items": {
"type": "string"
},
"description": "Producer snake_case reason codes explaining a withheld measurement, present when metrics.power_valid == 0. Absent on legacy rows and validated rows."
},
"power_audit": {
"type": "object",
"properties": {
"window_start_unix": {
"type": "number"
},
"window_end_unix": {
"type": "number"
},
"expected_gpu_count": {
"type": "integer"
},
"observed_gpu_count": {
"type": "integer"
},
"sample_count": {
"type": "integer"
},
"max_sample_gap_s": {
"type": "number"
},
"producer_sha": {
"type": [
"string",
"null"
]
},
"exporter_image_sha256": {
"type": [
"string",
"null"
]
},
"source": {
"type": "string"
},
"observed_gpu_ids": {
"type": "array",
"items": {
"type": "string"
}
},
"cpu": {
"type": "object",
"properties": {
"sensor_kind": {
"type": "string",
"enum": [
"module",
"grace_socket",
"dcgm_cpu_rail",
"rapl_package"
]
},
"source": {
"type": "string",
"description": "Collector that produced the CPU-side samples: acpi (hwmon), dcgm, or rapl (AMD EPYC package energy counter)."
},
"expected_sockets": {
"type": "integer"
},
"observed_sockets": {
"type": "integer"
},
"sample_row_count": {
"type": "integer"
},
"reason_codes": {
"type": "array",
"items": {
"type": "string"
}
}
},
"additionalProperties": true,
"description": "CPU-side leg provenance emitted alongside cpu_power_valid: the sensor kind that fed the CPU-side keys (on NVL72, module preferred over grace_socket over dcgm_cpu_rail; rapl_package on single-node AMD EPYC hosts), its collector, expected vs. observed sockets (two per compute tray or AMD host), stored sample rows, and the leg’s reason codes (empty when valid). Absent on runs without the CPU-side leg."
}
},
"additionalProperties": true,
"description": "Compact power measurement-window audit emitted alongside the power_valid verdict: window bounds, expected vs. observed GPU counts, sample statistics, and producer identity (producer_sha / exporter_image_sha256 are null for single-node telemetry without an srt-slurm producer). source names the validation document: a relative path within the source run artifact bundle or, on AgentX rows whose retained window ingest or backfill recovered, the alias power_validation_<stem>_conc<N>.json for LOGS/agentic/conc_<N>/power_validation.json in the power_audit_<stem> artifact (such rows carry only source and the window bounds); observed_gpu_ids contains producer device identifiers, which may be indices rather than physical UUIDs. cpu carries the CPU-side leg’s provenance (NVL72 or single-node AMD) when that leg ran. Present on valid and invalid rows when emitted; absence alone does not establish age or validity."
},
"date": {
"type": "string",
"format": "date"
},
"workflow_run_id": {
"type": "integer"
},
"run_started_at": {
"type": [
"string",
"null"
],
"format": "date-time"
},
"run_url": {
"type": [
"string",
"null"
]
}
},
"required": [
"id",
"hardware",
"framework",
"model",
"precision",
"spec_method",
"disagg",
"is_multinode",
"prefill_tp",
"prefill_ep",
"prefill_dp_attention",
"prefill_num_workers",
"decode_tp",
"decode_ep",
"decode_dp_attention",
"decode_num_workers",
"num_prefill_gpu",
"num_decode_gpu",
"benchmark_type",
"isl",
"osl",
"conc",
"offload_mode",
"image",
"recipe_fingerprint",
"metrics",
"date",
"run_url"
],
"additionalProperties": false
}
}Scroll to inspect the full example
Response example
[
{
"id": 421,
"hardware": "h200_sxm",
"framework": "vllm",
"model": "dsr1",
"precision": "fp8",
"spec_method": "none",
"disagg": false,
"is_multinode": false,
"prefill_tp": 8,
"prefill_ep": 1,
"prefill_dp_attention": false,
"prefill_num_workers": 1,
"decode_tp": 8,
"decode_ep": 1,
"decode_dp_attention": false,
"decode_num_workers": 1,
"num_prefill_gpu": 0,
"num_decode_gpu": 8,
"benchmark_type": "single_turn",
"isl": 1024,
"osl": 1024,
"conc": 32,
"offload_mode": "off",
"image": "vllm/vllm-openai:v0.10.2",
"recipe_fingerprint": "7d72a33d7d72a33d7d72a33d7d72a33d7d72a33d7d72a33d7d72a33d7d72a33d",
"metrics": {
"median_ttft": 0.42,
"median_tpot": 0.018,
"tput_per_gpu": 128.4,
"power_valid": 1,
"power_metric_schema_version": 2,
"avg_power_w": 678.5,
"joules_per_output_token": 5.3,
"joules_per_total_token": 2.65,
"avg_temp_c": 61.2
},
"date": "2026-08-08",
"run_url": "https://github.com/semianalysis/inference-benchmarks/actions/runs/123456789"
}
]Scroll to inspect the full example
400
Required parameters are missing or the model is unsupported.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"error": {
"type": "string"
}
},
"required": [
"error"
],
"additionalProperties": true
}Response example
{
"error": "model, isl, and osl are required"
}500
The history query failed.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"error": {
"type": "string"
}
},
"required": [
"error"
],
"additionalProperties": true
}Response example
{
"error": "Internal server error"
}GETStable/api/v1/workflow-infoRead workflow provenance
Returns workflow runs, changelogs, available configurations, and per-run configuration coverage for one date. date is required.
Parameters
| Name | Location | Type | Requirement | Description | Example |
|---|---|---|---|---|---|
| date | query | date | Required | Required YYYY-MM-DD date. | 2026-08-08 |
| benchmarkType | query | string | Optional | Set to agentic_traces to scope per-run configuration coverage to Agentic Traces. | agentic_traces |
Request
curl
curl -sS 'https://inferencex.semianalysis.com/api/v1/workflow-info?date=2026-08-08'Responses
WorkflowInfo200
Workflow provenance grouped into four arrays.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"runs": {
"type": "array",
"items": {
"type": "object",
"properties": {
"github_run_id": {
"type": "integer"
},
"name": {
"type": "string"
},
"conclusion": {
"type": [
"string",
"null"
]
},
"run_attempt": {
"type": "integer"
},
"html_url": {
"type": [
"string",
"null"
]
},
"created_at": {
"type": "string",
"format": "date-time"
},
"date": {
"type": "string",
"format": "date"
}
},
"required": [
"github_run_id",
"name",
"conclusion",
"run_attempt",
"html_url",
"created_at",
"date"
],
"additionalProperties": false
}
},
"changelogs": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": true
}
},
"configs": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": true
}
},
"runConfigs": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": true
}
}
},
"required": [
"runs",
"changelogs",
"configs",
"runConfigs"
],
"additionalProperties": false
}Scroll to inspect the full example
Response example
{
"runs": [
{
"github_run_id": 123456789,
"name": "nightly-h200",
"conclusion": "success",
"run_attempt": 1,
"html_url": "https://github.com/semianalysis/inference-benchmarks/actions/runs/123456789",
"created_at": "2026-08-08T03:00:00Z",
"date": "2026-08-08"
}
],
"changelogs": [],
"configs": [],
"runConfigs": []
}Scroll to inspect the full example
400
date is missing or not YYYY-MM-DD.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"error": {
"type": "string"
}
},
"required": [
"error"
],
"additionalProperties": true
}Response example
{
"error": "Invalid date format (YYYY-MM-DD required)"
}500
The workflow query failed.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"error": {
"type": "string"
}
},
"required": [
"error"
],
"additionalProperties": true
}Response example
{
"error": "Internal server error"
}GETStable/api/v1/evaluationsList evaluation aggregates
Returns latest-attempt evaluation results with configuration, task, provenance, and metric values.
Parameters
No parameters
Request
curl
curl -sS 'https://inferencex.semianalysis.com/api/v1/evaluations'Responses
EvaluationRows200
Evaluation result rows.
Media type: application/jsonResponse shape
{
"type": "array",
"items": {
"type": "object",
"properties": {
"id": {
"type": "integer"
},
"config_id": {
"type": "integer"
},
"hardware": {
"type": "string"
},
"framework": {
"type": "string"
},
"model": {
"type": "string"
},
"precision": {
"type": "string"
},
"spec_method": {
"type": "string"
},
"spec_algorithm": {
"type": [
"string",
"null"
],
"description": "Speculative algorithm recorded by a benchmark result for the same config and workflow run, or null when none was recorded."
},
"spec_draft_model": {
"type": [
"string",
"null"
],
"description": "Draft model recorded alongside spec_algorithm (\"native\" for built-in heads), or null when none was recorded."
},
"disagg": {
"type": "boolean"
},
"is_multinode": {
"type": "boolean"
},
"prefill_tp": {
"type": "integer"
},
"prefill_ep": {
"type": "integer"
},
"prefill_dp_attention": {
"type": "boolean"
},
"prefill_num_workers": {
"type": "integer"
},
"decode_tp": {
"type": "integer"
},
"decode_ep": {
"type": "integer"
},
"decode_dp_attention": {
"type": "boolean"
},
"decode_num_workers": {
"type": "integer"
},
"num_prefill_gpu": {
"type": "integer",
"description": "Physical prefill chips; aggregate engines may mirror their single chip count in both role columns."
},
"num_decode_gpu": {
"type": "integer",
"description": "Physical decode chips, independent of logical TP and DP. Sum role counts only for disaggregated engines."
},
"task": {
"type": "string"
},
"date": {
"type": "string",
"format": "date"
},
"conc": {
"type": [
"number",
"null"
]
},
"metrics": {
"type": "object",
"additionalProperties": {
"type": "number"
}
},
"timestamp": {
"type": "string",
"format": "date-time"
},
"run_url": {
"type": [
"string",
"null"
]
}
},
"required": [
"id",
"config_id",
"hardware",
"framework",
"model",
"precision",
"spec_method",
"spec_algorithm",
"spec_draft_model",
"disagg",
"is_multinode",
"prefill_tp",
"prefill_ep",
"prefill_dp_attention",
"prefill_num_workers",
"decode_tp",
"decode_ep",
"decode_dp_attention",
"decode_num_workers",
"num_prefill_gpu",
"num_decode_gpu",
"task",
"date",
"conc",
"metrics",
"timestamp",
"run_url"
],
"additionalProperties": false
}
}Scroll to inspect the full example
Response example
[
{
"id": 72,
"config_id": 11,
"hardware": "h200_sxm",
"framework": "vllm",
"model": "dsr1",
"precision": "fp8",
"spec_method": "none",
"spec_algorithm": null,
"spec_draft_model": null,
"disagg": false,
"is_multinode": false,
"prefill_tp": 8,
"prefill_ep": 1,
"prefill_dp_attention": false,
"prefill_num_workers": 1,
"decode_tp": 8,
"decode_ep": 1,
"decode_dp_attention": false,
"decode_num_workers": 1,
"num_prefill_gpu": 0,
"num_decode_gpu": 8,
"task": "gpqa",
"date": "2026-08-08",
"conc": null,
"metrics": {
"accuracy": 0.78
},
"timestamp": "2026-08-08T03:00:00Z",
"run_url": null
}
]Scroll to inspect the full example
500
The evaluation query failed.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"error": {
"type": "string"
}
},
"required": [
"error"
],
"additionalProperties": true
}Response example
{
"error": "Internal server error"
}GETStable/api/v1/reliabilityList benchmark reliability
Returns successful and total run counts by hardware and date.
Parameters
No parameters
Request
curl
curl -sS 'https://inferencex.semianalysis.com/api/v1/reliability'Responses
ReliabilityRows200
Reliability count rows.
Media type: application/jsonResponse shape
{
"type": "array",
"items": {
"type": "object",
"properties": {
"hardware": {
"type": "string"
},
"date": {
"type": "string",
"format": "date"
},
"n_success": {
"type": "integer"
},
"total": {
"type": "integer"
}
},
"required": [
"hardware",
"date",
"n_success",
"total"
],
"additionalProperties": false
}
}Scroll to inspect the full example
Response example
[
{
"hardware": "h200_sxm",
"date": "2026-08-08",
"n_success": 18,
"total": 20
}
]500
The reliability query failed.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"error": {
"type": "string"
}
},
"required": [
"error"
],
"additionalProperties": true
}Response example
{
"error": "Internal server error"
}External feeds
Stable feeds for spreadsheets, release tracking, submissions, and runtime images.
GETBeta/api/v1/paretoRead Pareto frontier and hinterland
Computes both global boundaries over stored benchmark observations using the chart’s dominance algorithm. Frontier uses the requested axis preferences; hinterland reverses both. Coordinates are raw metrics (no display-unit conversion), ordered by ascending x; observations preserves all ties and producer/snapshot fields. No interpolation, shaded geometry or scenery is returned. Missing/non-finite axes are excluded and counted; zero and negative finite values remain valid. The selected workload and raw model are required. This is not a reproduction of browser-only Optimal Only, quick filters, hidden series, log-axis exclusions or unofficial artifact overlays. Power axes require powerValid=strictV2. Empty selections return 200 with empty boundaries. Chart Advanced has a separate Pareto Frontier control: i_frontier=1 means plain green shading, 2 means scenic background; omit to disable. Retired i_hinterland share params are ignored by the dashboard and rejected here. Do not pass chart UI parameters to this endpoint.
Parameters
| Name | Location | Type | Requirement | Description | Example |
|---|---|---|---|---|---|
| model | query | string | Required | Display model name. | DeepSeek-R1-0528 |
| rawModel | query | string | Required | Exact stored model key belonging to model; prevents pooling variants. | dsr1 |
| sequence | query | enum | Required | Exact workload. AgentX uses benchmark_type=agentic_traces; fixed workloads use single_turn and matching ISL/OSL. | 1k/1k |
| xMetric | query | string | Required | Raw metrics object key, not a chart label. Unknown or absent keys produce no eligible observations. Units follow BenchmarkRow.metrics. | median_intvty |
| yMetric | query | string | Required | Raw metrics object key, not a chart label. Unknown or absent keys produce no eligible observations. Units follow BenchmarkRow.metrics. | output_tput_per_gpu |
| xDirection | query | enum | Required | Explicit preference: max for higher-is-better, min for lower-is-better. | max |
| yDirection | query | enum | Required | Explicit preference: max for higher-is-better, min for lower-is-better. | max |
| hardware | query | string | Optional | One exact stored key. Omit to pool all matching values; unknown values produce an empty selection. | h200_sxm |
| framework | query | string | Optional | One exact stored key. Omit to pool all matching values; unknown values produce an empty selection. | vllm |
| precision | query | string | Optional | One exact stored key. Omit to pool all matching values; unknown values produce an empty selection. | fp8 |
| date | query | date | Optional | As-of snapshot date; exact=true selects that snapshot date. Observation dates can be older. Cannot accompany exactRun=true. | 2026-08-08 |
| exact | query | boolean | Optional | Requires date when true. Only literal true/false accepted. | false |
| runId | query | string | Optional | Positive safe-integer GitHub Actions run ID. Constrains the latest lookup unless exactRun=true; malformed values are rejected. | 123456789 |
| exactRun | query | boolean | Optional | With runId, read its logical stored snapshot. Append-only runs include predecessors whose shared topologies keep the same image. Cannot accompany date or exact. Only literal true/false accepted. | false |
| powerValid | query | enum | Optional | strictV2 is required for registered power/energy metric axes; optional otherwise. Filters raw rows before selection and counting. | strictV2 |
Request
curl
curl -sS 'https://inferencex.semianalysis.com/api/v1/pareto?model=DeepSeek-R1-0528&rawModel=dsr1&sequence=1k%2F1k&xMetric=median_intvty&yMetric=output_tput_per_gpu&xDirection=max&yDirection=max'Responses
ParetoBoundaries200
Boundary coordinates and source observations; empty results are valid.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"source_url": {
"type": "string",
"description": "Relative raw benchmark request URL, including snapshot and power selectors. Resolve against the API origin."
},
"selection": {
"type": "object",
"properties": {
"model": {
"type": "string"
},
"rawModel": {
"type": "string"
},
"sequence": {
"type": "string",
"enum": [
"1k/1k",
"1k/8k",
"8k/1k",
"agentic-traces"
]
},
"xMetric": {
"type": "string"
},
"yMetric": {
"type": "string"
},
"xDirection": {
"type": "string",
"enum": [
"min",
"max"
]
},
"yDirection": {
"type": "string",
"enum": [
"min",
"max"
]
},
"hardware": {
"type": [
"string",
"null"
]
},
"framework": {
"type": [
"string",
"null"
]
},
"precision": {
"type": [
"string",
"null"
]
}
},
"required": [
"model",
"rawModel",
"sequence",
"xMetric",
"yMetric",
"xDirection",
"yDirection",
"hardware",
"framework",
"precision"
],
"additionalProperties": false
},
"counts": {
"type": "object",
"properties": {
"returned": {
"type": "integer",
"description": "Rows from the raw request after any powerValid filter."
},
"selected": {
"type": "integer",
"description": "Rows after raw-model, workload, hardware, framework and precision selection."
},
"eligible": {
"type": "integer",
"description": "Selected observations with both finite numeric axes, including coordinate ties."
},
"missing_or_nonfinite": {
"type": "integer",
"description": "Selected observations excluded for absent or non-finite axes."
}
},
"required": [
"returned",
"selected",
"eligible",
"missing_or_nonfinite"
],
"additionalProperties": false
},
"frontier": {
"type": "array",
"items": {
"type": "object",
"properties": {
"x": {
"type": "number"
},
"y": {
"type": "number"
},
"observations": {
"type": "array",
"items": {
"type": "object",
"properties": {
"id": {
"type": "integer"
},
"hardware": {
"type": "string"
},
"framework": {
"type": "string"
},
"model": {
"type": "string"
},
"precision": {
"type": "string"
},
"spec_method": {
"type": "string"
},
"disagg": {
"type": "boolean"
},
"is_multinode": {
"type": "boolean"
},
"prefill_tp": {
"type": "integer"
},
"prefill_ep": {
"type": "integer"
},
"prefill_dp_attention": {
"type": "boolean"
},
"prefill_num_workers": {
"type": "integer"
},
"decode_tp": {
"type": "integer"
},
"decode_ep": {
"type": "integer"
},
"decode_dp_attention": {
"type": "boolean"
},
"decode_num_workers": {
"type": "integer"
},
"num_prefill_gpu": {
"type": "integer",
"description": "Physical prefill chips; aggregate engines may mirror their single chip count in both role columns."
},
"num_decode_gpu": {
"type": "integer",
"description": "Physical decode chips, independent of logical TP and DP. Sum role counts only for disaggregated engines."
},
"benchmark_type": {
"type": "string"
},
"isl": {
"type": [
"number",
"null"
]
},
"osl": {
"type": [
"number",
"null"
]
},
"conc": {
"type": "integer"
},
"offload_mode": {
"type": "string"
},
"image": {
"type": [
"string",
"null"
]
},
"recipe_fingerprint": {
"type": [
"string",
"null"
]
},
"metrics": {
"type": "object",
"additionalProperties": {
"type": "number"
},
"description": "Scalar metric map. Time metrics, including p99_itl and p99_tpot, are in seconds. p99_itl measures inter-token latency; p99_tpot measures per-request time per output token. Use the actual p99_itl field for an inter-token latency requirement, not the reciprocal of p99_intvty. Throughput metrics use tokens per second per GPU unless their name states otherwise; output_tput_per_gpu counts output tokens. On disaggregated rows tput_per_gpu divides by every chip, input_tput_per_gpu by prefill chips and output_tput_per_gpu by decode chips, for fixed-sequence and agentic runs alike. Agentic disaggregated rows also keep the original per-total-chip rates from the producer as input_tput_per_total_gpu and output_tput_per_total_gpu. Keys evolve independently; measured power / energy / GPU-telemetry keys are typed below.",
"properties": {
"power_valid": {
"type": "number",
"description": "Publication verdict: 1 = validated measurement window; 0 = failed validation — measured power/energy values are withheld from this row end-to-end, so treat any that remain as unreliable; absent = no validation verdict is available in this response. Absence alone establishes neither the reason, the measurement age, nor invalidity."
},
"power_metric_schema_version": {
"type": "number",
"description": "Power schema version. Version 2 defines every unprefixed joules_per_* field as whole-deployment energy, including on disaggregated runs."
},
"cpu_power_valid": {
"type": "number",
"description": "CPU-side leg verdict (NVL72 Grace sockets or single-node AMD EPYC RAPL packages), independent of power_valid: 1 = the CPU-side (and module, when present) keys were integrated over the validated window with full socket coverage; 0 = the leg failed and no CPU-side keys are emitted; absent = the run had no CPU-side leg. Ingest withholds the CPU-side keys unless this verdict is 1, and power_valid = 0 does not remove them. GPU power and energy never change because of this verdict."
},
"avg_power_w": {
"type": "number",
"description": "Mean per-GPU power draw in watts during the measured load window."
},
"avg_total_gpu_power_w": {
"type": "number",
"description": "Mean total GPU power draw in watts across the measured deployment."
},
"total_gpu_energy_j": {
"type": "number",
"description": "Total GPU energy integrated over the measured deployment load window, in joules."
},
"p75_power_w": {
"type": "number",
"description": "Time-weighted P75 of synchronized total GPU-board watts, divided by participating GPU count, over the validated benchmark window with piecewise-linear interpolation. Not a percentile of individual-device percentiles; absent when not measured."
},
"p75_total_gpu_power_w": {
"type": "number",
"description": "Time-weighted P75 of synchronized total GPU-board watts over the validated benchmark window with piecewise-linear interpolation. Accelerator boards only, not chassis or facility power."
},
"p90_power_w": {
"type": "number",
"description": "Time-weighted P90 of synchronized total GPU-board watts, divided by participating GPU count, over the validated benchmark window with piecewise-linear interpolation. Not a percentile of individual-device percentiles; absent when not measured."
},
"p90_total_gpu_power_w": {
"type": "number",
"description": "Time-weighted P90 of synchronized total GPU-board watts over the validated benchmark window with piecewise-linear interpolation. Accelerator boards only, not chassis or facility power."
},
"joules_per_successful_query": {
"type": "number",
"description": "Whole-deployment energy in joules divided by successful requests."
},
"joules_per_output_token": {
"type": "number",
"description": "Energy per generated output token in joules; cluster-wide on schema-version-2 rows, including disaggregated runs."
},
"joules_per_total_token": {
"type": "number",
"description": "Total system energy divided by input plus output tokens; a workload-shape-fair view that does not treat prompt tokens as free."
},
"prefill_avg_power_w": {
"type": "number",
"description": "Mean per-GPU power draw in watts across prefill workers; emitted only for deployments with distinct prefill and decode roles."
},
"decode_avg_power_w": {
"type": "number",
"description": "Mean per-GPU power draw in watts across decode workers; emitted only for deployments with distinct prefill and decode roles."
},
"joules_per_input_token": {
"type": "number",
"description": "Energy per input token in joules; cluster-wide on schema-version-2 rows."
},
"prefill_joules_per_input_token": {
"type": "number",
"description": "Role-local prefill energy per input token in joules."
},
"decode_joules_per_output_token": {
"type": "number",
"description": "Role-local decode energy per generated output token in joules."
},
"avg_temp_c": {
"type": "number",
"description": "Mean per-GPU temperature in degrees Celsius during the load window."
},
"peak_temp_c": {
"type": "number",
"description": "Maximum instantaneous per-GPU temperature in degrees Celsius during the load window."
},
"avg_util_pct": {
"type": "number",
"description": "Mean per-GPU utilization percentage (0-100) during the load window."
},
"avg_mem_used_mb": {
"type": "number",
"description": "Mean per-GPU memory used in MB during the load window."
},
"avg_cpu_socket_power_w": {
"type": "number",
"description": "Mean over Grace sockets of each socket’s window-mean Grace-side power in watts (ACPI hwmon Grace Power Socket sensor, which NVIDIA defines only as the Grace socket power; modeled as the CPU and SysIO rails, LPDDR5X and socket regulation) over the same validated window as GPU energy. On single-node AMD rows (power_audit.cpu.sensor_kind = rapl_package) it is instead each EPYC package’s RAPL energy-counter power, which has no DRAM domain."
},
"avg_total_cpu_power_w": {
"type": "number",
"description": "Sum over every CPU socket in the deployment of window-mean socket power in watts (Grace side on NVL72, RAPL package on single-node AMD); dividing by avg_cpu_socket_power_w recovers the socket count (two per compute tray or AMD host)."
},
"total_cpu_energy_j": {
"type": "number",
"description": "CPU-side energy in joules (Grace side on NVL72, RAPL package on single-node AMD) integrated over the validated window across every socket. Additive to total_gpu_energy_j, never included in it."
},
"avg_total_module_power_w": {
"type": "number",
"description": "Sum over sockets of window-mean compute-module power in watts (Grace + its Blackwell GPUs + HBM + LPDDR5X + regulator loss, ACPI hwmon Module Power Socket sensor); emitted only when the module sensor exists on every socket. It already contains the GPU-board watts, so do not add avg_total_gpu_power_w to it."
},
"total_module_energy_j": {
"type": "number",
"description": "Compute-module energy in joules over the validated window across every socket; emitted with avg_total_module_power_w and includes the GPU energy."
},
"avg_total_cpu_rail_power_w": {
"type": "number",
"description": "Sum over Grace sockets of window-mean CPU core rail power in watts (ACPI CPU Power Socket sensor, no SysIO); emitted only when every socket reports a gap-free rail. A breakdown inside avg_total_cpu_power_w, not additive to it. NVL72 only."
},
"total_cpu_rail_energy_j": {
"type": "number",
"description": "CPU core rail energy in joules over the validated window across every socket; emitted with avg_total_cpu_rail_power_w and already included in total_cpu_energy_j."
},
"avg_total_cpu_sysio_power_w": {
"type": "number",
"description": "Sum over Grace sockets of window-mean SysIO rail power in watts (ACPI SysIO Power Socket sensor); emitted only when every socket reports a gap-free rail. A breakdown inside avg_total_cpu_power_w, not additive to it. NVL72 only."
},
"total_cpu_sysio_energy_j": {
"type": "number",
"description": "SysIO rail energy in joules over the validated window across every socket; emitted with avg_total_cpu_sysio_power_w and already included in total_cpu_energy_j."
}
}
},
"workers": {
"type": "array",
"items": {
"type": "object",
"properties": {
"role": {
"type": "string"
},
"worker_idx": {
"type": "integer"
},
"hosts": {
"type": "array",
"items": {
"type": "string"
}
},
"num_gpus": {
"type": "integer"
},
"avg_power_w": {
"type": "number"
},
"avg_temp_c": {
"type": "number"
},
"peak_temp_c": {
"type": "number"
},
"avg_util_pct": {
"type": "number"
},
"avg_mem_used_mb": {
"type": "number"
}
},
"required": [
"role",
"worker_idx",
"num_gpus",
"avg_power_w"
],
"additionalProperties": false
}
},
"power_invalid_reasons": {
"type": "array",
"items": {
"type": "string"
},
"description": "Producer snake_case reason codes explaining a withheld measurement, present when metrics.power_valid == 0. Absent on legacy rows and validated rows."
},
"power_audit": {
"type": "object",
"properties": {
"window_start_unix": {
"type": "number"
},
"window_end_unix": {
"type": "number"
},
"expected_gpu_count": {
"type": "integer"
},
"observed_gpu_count": {
"type": "integer"
},
"sample_count": {
"type": "integer"
},
"max_sample_gap_s": {
"type": "number"
},
"producer_sha": {
"type": [
"string",
"null"
]
},
"exporter_image_sha256": {
"type": [
"string",
"null"
]
},
"source": {
"type": "string"
},
"observed_gpu_ids": {
"type": "array",
"items": {
"type": "string"
}
},
"cpu": {
"type": "object",
"properties": {
"sensor_kind": {
"type": "string",
"enum": [
"module",
"grace_socket",
"dcgm_cpu_rail",
"rapl_package"
]
},
"source": {
"type": "string",
"description": "Collector that produced the CPU-side samples: acpi (hwmon), dcgm, or rapl (AMD EPYC package energy counter)."
},
"expected_sockets": {
"type": "integer"
},
"observed_sockets": {
"type": "integer"
},
"sample_row_count": {
"type": "integer"
},
"reason_codes": {
"type": "array",
"items": {
"type": "string"
}
}
},
"additionalProperties": true,
"description": "CPU-side leg provenance emitted alongside cpu_power_valid: the sensor kind that fed the CPU-side keys (on NVL72, module preferred over grace_socket over dcgm_cpu_rail; rapl_package on single-node AMD EPYC hosts), its collector, expected vs. observed sockets (two per compute tray or AMD host), stored sample rows, and the leg’s reason codes (empty when valid). Absent on runs without the CPU-side leg."
}
},
"additionalProperties": true,
"description": "Compact power measurement-window audit emitted alongside the power_valid verdict: window bounds, expected vs. observed GPU counts, sample statistics, and producer identity (producer_sha / exporter_image_sha256 are null for single-node telemetry without an srt-slurm producer). source names the validation document: a relative path within the source run artifact bundle or, on AgentX rows whose retained window ingest or backfill recovered, the alias power_validation_<stem>_conc<N>.json for LOGS/agentic/conc_<N>/power_validation.json in the power_audit_<stem> artifact (such rows carry only source and the window bounds); observed_gpu_ids contains producer device identifiers, which may be indices rather than physical UUIDs. cpu carries the CPU-side leg’s provenance (NVL72 or single-node AMD) when that leg ran. Present on valid and invalid rows when emitted; absence alone does not establish age or validity."
},
"date": {
"type": "string",
"format": "date"
},
"workflow_run_id": {
"type": "integer"
},
"run_started_at": {
"type": [
"string",
"null"
],
"format": "date-time"
},
"run_url": {
"type": [
"string",
"null"
]
}
},
"required": [
"id",
"hardware",
"framework",
"model",
"precision",
"spec_method",
"disagg",
"is_multinode",
"prefill_tp",
"prefill_ep",
"prefill_dp_attention",
"prefill_num_workers",
"decode_tp",
"decode_ep",
"decode_dp_attention",
"decode_num_workers",
"num_prefill_gpu",
"num_decode_gpu",
"benchmark_type",
"isl",
"osl",
"conc",
"offload_mode",
"image",
"recipe_fingerprint",
"metrics",
"date",
"run_url"
],
"additionalProperties": false
}
}
},
"required": [
"x",
"y",
"observations"
],
"additionalProperties": false
}
},
"hinterland": {
"type": "array",
"items": {
"type": "object",
"properties": {
"x": {
"type": "number"
},
"y": {
"type": "number"
},
"observations": {
"type": "array",
"items": {
"type": "object",
"properties": {
"id": {
"type": "integer"
},
"hardware": {
"type": "string"
},
"framework": {
"type": "string"
},
"model": {
"type": "string"
},
"precision": {
"type": "string"
},
"spec_method": {
"type": "string"
},
"disagg": {
"type": "boolean"
},
"is_multinode": {
"type": "boolean"
},
"prefill_tp": {
"type": "integer"
},
"prefill_ep": {
"type": "integer"
},
"prefill_dp_attention": {
"type": "boolean"
},
"prefill_num_workers": {
"type": "integer"
},
"decode_tp": {
"type": "integer"
},
"decode_ep": {
"type": "integer"
},
"decode_dp_attention": {
"type": "boolean"
},
"decode_num_workers": {
"type": "integer"
},
"num_prefill_gpu": {
"type": "integer",
"description": "Physical prefill chips; aggregate engines may mirror their single chip count in both role columns."
},
"num_decode_gpu": {
"type": "integer",
"description": "Physical decode chips, independent of logical TP and DP. Sum role counts only for disaggregated engines."
},
"benchmark_type": {
"type": "string"
},
"isl": {
"type": [
"number",
"null"
]
},
"osl": {
"type": [
"number",
"null"
]
},
"conc": {
"type": "integer"
},
"offload_mode": {
"type": "string"
},
"image": {
"type": [
"string",
"null"
]
},
"recipe_fingerprint": {
"type": [
"string",
"null"
]
},
"metrics": {
"type": "object",
"additionalProperties": {
"type": "number"
},
"description": "Scalar metric map. Time metrics, including p99_itl and p99_tpot, are in seconds. p99_itl measures inter-token latency; p99_tpot measures per-request time per output token. Use the actual p99_itl field for an inter-token latency requirement, not the reciprocal of p99_intvty. Throughput metrics use tokens per second per GPU unless their name states otherwise; output_tput_per_gpu counts output tokens. On disaggregated rows tput_per_gpu divides by every chip, input_tput_per_gpu by prefill chips and output_tput_per_gpu by decode chips, for fixed-sequence and agentic runs alike. Agentic disaggregated rows also keep the original per-total-chip rates from the producer as input_tput_per_total_gpu and output_tput_per_total_gpu. Keys evolve independently; measured power / energy / GPU-telemetry keys are typed below.",
"properties": {
"power_valid": {
"type": "number",
"description": "Publication verdict: 1 = validated measurement window; 0 = failed validation — measured power/energy values are withheld from this row end-to-end, so treat any that remain as unreliable; absent = no validation verdict is available in this response. Absence alone establishes neither the reason, the measurement age, nor invalidity."
},
"power_metric_schema_version": {
"type": "number",
"description": "Power schema version. Version 2 defines every unprefixed joules_per_* field as whole-deployment energy, including on disaggregated runs."
},
"cpu_power_valid": {
"type": "number",
"description": "CPU-side leg verdict (NVL72 Grace sockets or single-node AMD EPYC RAPL packages), independent of power_valid: 1 = the CPU-side (and module, when present) keys were integrated over the validated window with full socket coverage; 0 = the leg failed and no CPU-side keys are emitted; absent = the run had no CPU-side leg. Ingest withholds the CPU-side keys unless this verdict is 1, and power_valid = 0 does not remove them. GPU power and energy never change because of this verdict."
},
"avg_power_w": {
"type": "number",
"description": "Mean per-GPU power draw in watts during the measured load window."
},
"avg_total_gpu_power_w": {
"type": "number",
"description": "Mean total GPU power draw in watts across the measured deployment."
},
"total_gpu_energy_j": {
"type": "number",
"description": "Total GPU energy integrated over the measured deployment load window, in joules."
},
"p75_power_w": {
"type": "number",
"description": "Time-weighted P75 of synchronized total GPU-board watts, divided by participating GPU count, over the validated benchmark window with piecewise-linear interpolation. Not a percentile of individual-device percentiles; absent when not measured."
},
"p75_total_gpu_power_w": {
"type": "number",
"description": "Time-weighted P75 of synchronized total GPU-board watts over the validated benchmark window with piecewise-linear interpolation. Accelerator boards only, not chassis or facility power."
},
"p90_power_w": {
"type": "number",
"description": "Time-weighted P90 of synchronized total GPU-board watts, divided by participating GPU count, over the validated benchmark window with piecewise-linear interpolation. Not a percentile of individual-device percentiles; absent when not measured."
},
"p90_total_gpu_power_w": {
"type": "number",
"description": "Time-weighted P90 of synchronized total GPU-board watts over the validated benchmark window with piecewise-linear interpolation. Accelerator boards only, not chassis or facility power."
},
"joules_per_successful_query": {
"type": "number",
"description": "Whole-deployment energy in joules divided by successful requests."
},
"joules_per_output_token": {
"type": "number",
"description": "Energy per generated output token in joules; cluster-wide on schema-version-2 rows, including disaggregated runs."
},
"joules_per_total_token": {
"type": "number",
"description": "Total system energy divided by input plus output tokens; a workload-shape-fair view that does not treat prompt tokens as free."
},
"prefill_avg_power_w": {
"type": "number",
"description": "Mean per-GPU power draw in watts across prefill workers; emitted only for deployments with distinct prefill and decode roles."
},
"decode_avg_power_w": {
"type": "number",
"description": "Mean per-GPU power draw in watts across decode workers; emitted only for deployments with distinct prefill and decode roles."
},
"joules_per_input_token": {
"type": "number",
"description": "Energy per input token in joules; cluster-wide on schema-version-2 rows."
},
"prefill_joules_per_input_token": {
"type": "number",
"description": "Role-local prefill energy per input token in joules."
},
"decode_joules_per_output_token": {
"type": "number",
"description": "Role-local decode energy per generated output token in joules."
},
"avg_temp_c": {
"type": "number",
"description": "Mean per-GPU temperature in degrees Celsius during the load window."
},
"peak_temp_c": {
"type": "number",
"description": "Maximum instantaneous per-GPU temperature in degrees Celsius during the load window."
},
"avg_util_pct": {
"type": "number",
"description": "Mean per-GPU utilization percentage (0-100) during the load window."
},
"avg_mem_used_mb": {
"type": "number",
"description": "Mean per-GPU memory used in MB during the load window."
},
"avg_cpu_socket_power_w": {
"type": "number",
"description": "Mean over Grace sockets of each socket’s window-mean Grace-side power in watts (ACPI hwmon Grace Power Socket sensor, which NVIDIA defines only as the Grace socket power; modeled as the CPU and SysIO rails, LPDDR5X and socket regulation) over the same validated window as GPU energy. On single-node AMD rows (power_audit.cpu.sensor_kind = rapl_package) it is instead each EPYC package’s RAPL energy-counter power, which has no DRAM domain."
},
"avg_total_cpu_power_w": {
"type": "number",
"description": "Sum over every CPU socket in the deployment of window-mean socket power in watts (Grace side on NVL72, RAPL package on single-node AMD); dividing by avg_cpu_socket_power_w recovers the socket count (two per compute tray or AMD host)."
},
"total_cpu_energy_j": {
"type": "number",
"description": "CPU-side energy in joules (Grace side on NVL72, RAPL package on single-node AMD) integrated over the validated window across every socket. Additive to total_gpu_energy_j, never included in it."
},
"avg_total_module_power_w": {
"type": "number",
"description": "Sum over sockets of window-mean compute-module power in watts (Grace + its Blackwell GPUs + HBM + LPDDR5X + regulator loss, ACPI hwmon Module Power Socket sensor); emitted only when the module sensor exists on every socket. It already contains the GPU-board watts, so do not add avg_total_gpu_power_w to it."
},
"total_module_energy_j": {
"type": "number",
"description": "Compute-module energy in joules over the validated window across every socket; emitted with avg_total_module_power_w and includes the GPU energy."
},
"avg_total_cpu_rail_power_w": {
"type": "number",
"description": "Sum over Grace sockets of window-mean CPU core rail power in watts (ACPI CPU Power Socket sensor, no SysIO); emitted only when every socket reports a gap-free rail. A breakdown inside avg_total_cpu_power_w, not additive to it. NVL72 only."
},
"total_cpu_rail_energy_j": {
"type": "number",
"description": "CPU core rail energy in joules over the validated window across every socket; emitted with avg_total_cpu_rail_power_w and already included in total_cpu_energy_j."
},
"avg_total_cpu_sysio_power_w": {
"type": "number",
"description": "Sum over Grace sockets of window-mean SysIO rail power in watts (ACPI SysIO Power Socket sensor); emitted only when every socket reports a gap-free rail. A breakdown inside avg_total_cpu_power_w, not additive to it. NVL72 only."
},
"total_cpu_sysio_energy_j": {
"type": "number",
"description": "SysIO rail energy in joules over the validated window across every socket; emitted with avg_total_cpu_sysio_power_w and already included in total_cpu_energy_j."
}
}
},
"workers": {
"type": "array",
"items": {
"type": "object",
"properties": {
"role": {
"type": "string"
},
"worker_idx": {
"type": "integer"
},
"hosts": {
"type": "array",
"items": {
"type": "string"
}
},
"num_gpus": {
"type": "integer"
},
"avg_power_w": {
"type": "number"
},
"avg_temp_c": {
"type": "number"
},
"peak_temp_c": {
"type": "number"
},
"avg_util_pct": {
"type": "number"
},
"avg_mem_used_mb": {
"type": "number"
}
},
"required": [
"role",
"worker_idx",
"num_gpus",
"avg_power_w"
],
"additionalProperties": false
}
},
"power_invalid_reasons": {
"type": "array",
"items": {
"type": "string"
},
"description": "Producer snake_case reason codes explaining a withheld measurement, present when metrics.power_valid == 0. Absent on legacy rows and validated rows."
},
"power_audit": {
"type": "object",
"properties": {
"window_start_unix": {
"type": "number"
},
"window_end_unix": {
"type": "number"
},
"expected_gpu_count": {
"type": "integer"
},
"observed_gpu_count": {
"type": "integer"
},
"sample_count": {
"type": "integer"
},
"max_sample_gap_s": {
"type": "number"
},
"producer_sha": {
"type": [
"string",
"null"
]
},
"exporter_image_sha256": {
"type": [
"string",
"null"
]
},
"source": {
"type": "string"
},
"observed_gpu_ids": {
"type": "array",
"items": {
"type": "string"
}
},
"cpu": {
"type": "object",
"properties": {
"sensor_kind": {
"type": "string",
"enum": [
"module",
"grace_socket",
"dcgm_cpu_rail",
"rapl_package"
]
},
"source": {
"type": "string",
"description": "Collector that produced the CPU-side samples: acpi (hwmon), dcgm, or rapl (AMD EPYC package energy counter)."
},
"expected_sockets": {
"type": "integer"
},
"observed_sockets": {
"type": "integer"
},
"sample_row_count": {
"type": "integer"
},
"reason_codes": {
"type": "array",
"items": {
"type": "string"
}
}
},
"additionalProperties": true,
"description": "CPU-side leg provenance emitted alongside cpu_power_valid: the sensor kind that fed the CPU-side keys (on NVL72, module preferred over grace_socket over dcgm_cpu_rail; rapl_package on single-node AMD EPYC hosts), its collector, expected vs. observed sockets (two per compute tray or AMD host), stored sample rows, and the leg’s reason codes (empty when valid). Absent on runs without the CPU-side leg."
}
},
"additionalProperties": true,
"description": "Compact power measurement-window audit emitted alongside the power_valid verdict: window bounds, expected vs. observed GPU counts, sample statistics, and producer identity (producer_sha / exporter_image_sha256 are null for single-node telemetry without an srt-slurm producer). source names the validation document: a relative path within the source run artifact bundle or, on AgentX rows whose retained window ingest or backfill recovered, the alias power_validation_<stem>_conc<N>.json for LOGS/agentic/conc_<N>/power_validation.json in the power_audit_<stem> artifact (such rows carry only source and the window bounds); observed_gpu_ids contains producer device identifiers, which may be indices rather than physical UUIDs. cpu carries the CPU-side leg’s provenance (NVL72 or single-node AMD) when that leg ran. Present on valid and invalid rows when emitted; absence alone does not establish age or validity."
},
"date": {
"type": "string",
"format": "date"
},
"workflow_run_id": {
"type": "integer"
},
"run_started_at": {
"type": [
"string",
"null"
],
"format": "date-time"
},
"run_url": {
"type": [
"string",
"null"
]
}
},
"required": [
"id",
"hardware",
"framework",
"model",
"precision",
"spec_method",
"disagg",
"is_multinode",
"prefill_tp",
"prefill_ep",
"prefill_dp_attention",
"prefill_num_workers",
"decode_tp",
"decode_ep",
"decode_dp_attention",
"decode_num_workers",
"num_prefill_gpu",
"num_decode_gpu",
"benchmark_type",
"isl",
"osl",
"conc",
"offload_mode",
"image",
"recipe_fingerprint",
"metrics",
"date",
"run_url"
],
"additionalProperties": false
}
}
},
"required": [
"x",
"y",
"observations"
],
"additionalProperties": false
}
}
},
"required": [
"source_url",
"selection",
"counts",
"frontier",
"hinterland"
],
"additionalProperties": false
}Scroll to inspect the full example
Response example
{
"source_url": "/api/v1/benchmarks?model=DeepSeek-R1-0528",
"selection": {
"model": "DeepSeek-R1-0528",
"rawModel": "dsr1",
"sequence": "1k/1k",
"xMetric": "median_intvty",
"yMetric": "output_tput_per_gpu",
"xDirection": "max",
"yDirection": "max",
"hardware": null,
"framework": null,
"precision": null
},
"counts": {
"returned": 0,
"selected": 0,
"eligible": 0,
"missing_or_nonfinite": 0
},
"frontier": [],
"hinterland": []
}Scroll to inspect the full example
400
Missing, invalid, empty, repeated or unknown parameters, incompatible snapshot selectors, or power axes without strictV2.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"error": {
"type": "string"
}
},
"required": [
"error"
],
"additionalProperties": true
}Response example
{
"error": "Invalid xDirection"
}500
The source benchmark query or boundary computation failed.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"error": {
"type": "string"
}
},
"required": [
"error"
],
"additionalProperties": true
}Response example
{
"error": "Internal server error"
}GETStable/api/v1/tco-feedCompute a TCO feed
Computes Pareto-frontier throughput points or weighted scores for spreadsheet TCO models. Every scoring assumption is encoded in the URL. CSV returns the same selected view as a flat table.
Parameters
| Name | Location | Type | Requirement | Description | Example |
|---|---|---|---|---|---|
| model | query | string | Optional | DB model key or display model name. | dsv4 |
| workloads | query | CSV workload list | Optional | Comma-separated <isl>x<osl> token pairs. | 1024x1024,8192x1024 |
| tiers | query | CSV number list | Optional | Positive interactivity targets in output tokens per second per user. | 30,50,75,100 |
| date | query | date | Optional | Use data on or before YYYY-MM-DD. Omit for latest. | 2026-08-08 |
| format | query | enum | Optional | Response encoding. | json |
| view | query | enum | Optional | points returns one row per hardware, workload, and tier. scores returns one row per hardware. | points |
| weights | query | CSV number list | Optional | scores only. One non-negative weight per tier, normalized to sum to 1. | 0.35,0.4,0.2,0.05 |
| workload_weights | query | CSV number list | Optional | scores only. One non-negative weight per workload, normalized to sum to 1. Defaults to equal weights. | 0.5,0.5 |
| alpha | query | number | Optional | scores only. Input-token value ratio in [0, 10]. | 0.25 |
Request
curl
curl -sS 'https://inferencex.semianalysis.com/api/v1/tco-feed?model=dsv4&workloads=1024x1024,8192x1024&tiers=30,50,75,100&view=points&format=json'Responses
TcoFeed200
The selected points or scores envelope.
Media type: application/jsonResponse shape
{
"oneOf": [
{
"type": "object",
"properties": {
"model": {
"type": "string"
},
"db_model_keys": {
"type": "array",
"items": {
"type": "string"
}
},
"date": {
"type": [
"string",
"null"
],
"format": "date"
},
"workloads": {
"type": "array",
"items": {
"type": "string"
}
},
"tiers": {
"type": "array",
"items": {
"type": "number"
}
},
"rows": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": true
}
}
},
"required": [
"model",
"db_model_keys",
"date",
"workloads",
"tiers",
"rows"
],
"additionalProperties": false
},
{
"type": "object",
"properties": {
"model": {
"type": "string"
},
"db_model_keys": {
"type": "array",
"items": {
"type": "string"
}
},
"date": {
"type": [
"string",
"null"
],
"format": "date"
},
"workloads": {
"type": "array",
"items": {
"type": "string"
}
},
"tiers": {
"type": "array",
"items": {
"type": "number"
}
},
"weights": {
"type": "array",
"items": {
"type": "number"
}
},
"workload_weights": {
"type": "array",
"items": {
"type": "number"
}
},
"alpha": {
"type": "number"
},
"rows": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": true
}
}
},
"required": [
"model",
"db_model_keys",
"date",
"workloads",
"tiers",
"weights",
"workload_weights",
"alpha",
"rows"
],
"additionalProperties": false
}
]
}Scroll to inspect the full example
Response example
{
"model": "dsv4",
"db_model_keys": [
"dsv4"
],
"date": null,
"workloads": [
"1024x1024"
],
"tiers": [
50
],
"rows": [
{
"hardware": "h200_sxm",
"workload": "1024x1024",
"tier": 50,
"output_tput_per_gpu": 118.2,
"boundary": "interpolated",
"is_interpolated": true,
"frontier_points": 3,
"frontier_min_interactivity": 30,
"frontier_max_interactivity": 75,
"latest_date": "2026-08-08",
"oldest_frontier_date": "2026-08-06",
"evidence_date": {
"from": "2026-08-06",
"to": "2026-08-08"
}
}
]
}Scroll to inspect the full example
Alternate representation: text/csv
Response shape
{
"type": "string"
}Response example
hardware,workload,tier,output_tput_per_gpu,boundary,frontier_points,frontier_min_interactivity,frontier_max_interactivity,latest_date,oldest_frontier_date
h200_sxm,1024x1024,50,118.2,interpolated,3,30,75,2026-08-08,2026-08-06
400
A model, date, view, format, workload, tier, weight, or alpha value is invalid.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"error": {
"type": "string"
}
},
"required": [
"error"
],
"additionalProperties": true
}Response example
{
"error": "Invalid tiers: expected comma-separated positive numbers"
}500
The TCO calculation failed.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"error": {
"type": "string"
}
},
"required": [
"error"
],
"additionalProperties": true
}Response example
{
"error": "Internal server error"
}GETStable/api/v1/submissionsRead submission coverage
Returns configuration-level submission summaries and daily hardware submission volume.
Parameters
No parameters
Request
curl
curl -sS 'https://inferencex.semianalysis.com/api/v1/submissions'Responses
Submissions200
Submission summary and volume arrays.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"summary": {
"type": "array",
"items": {
"type": "object",
"properties": {
"model": {
"type": "string"
},
"hardware": {
"type": "string"
},
"framework": {
"type": "string"
},
"precision": {
"type": "string"
},
"spec_method": {
"type": "string"
},
"disagg": {
"type": "boolean"
},
"is_multinode": {
"type": "boolean"
},
"num_prefill_gpu": {
"type": "integer",
"description": "Physical prefill chips; aggregate engines may mirror their single chip count in both role columns."
},
"num_decode_gpu": {
"type": "integer",
"description": "Physical decode chips, independent of logical TP and DP. Sum role counts only for disaggregated engines."
},
"prefill_tp": {
"type": "integer"
},
"prefill_ep": {
"type": "integer"
},
"decode_tp": {
"type": "integer"
},
"decode_ep": {
"type": "integer"
},
"date": {
"type": "string",
"format": "date"
},
"total_datapoints": {
"type": "integer"
},
"distinct_sequences": {
"type": "integer"
},
"distinct_concurrencies": {
"type": "integer"
},
"max_concurrency": {
"type": "integer"
},
"image": {
"type": [
"string",
"null"
]
}
},
"required": [
"model",
"hardware",
"framework",
"precision",
"spec_method",
"disagg",
"is_multinode",
"num_prefill_gpu",
"num_decode_gpu",
"prefill_tp",
"prefill_ep",
"decode_tp",
"decode_ep",
"date",
"total_datapoints",
"distinct_sequences",
"distinct_concurrencies",
"max_concurrency",
"image"
],
"additionalProperties": false
}
},
"volume": {
"type": "array",
"items": {
"type": "object",
"properties": {
"date": {
"type": "string",
"format": "date"
},
"hardware": {
"type": "string"
},
"datapoints": {
"type": "integer"
}
},
"required": [
"date",
"hardware",
"datapoints"
],
"additionalProperties": false
}
}
},
"required": [
"summary",
"volume"
],
"additionalProperties": false
}Scroll to inspect the full example
Response example
{
"summary": [
{
"model": "dsr1",
"hardware": "h200_sxm",
"framework": "vllm",
"precision": "fp8",
"spec_method": "none",
"disagg": false,
"is_multinode": false,
"num_prefill_gpu": 0,
"num_decode_gpu": 8,
"prefill_tp": 1,
"prefill_ep": 1,
"decode_tp": 8,
"decode_ep": 1,
"date": "2026-08-08",
"total_datapoints": 24,
"distinct_sequences": 3,
"distinct_concurrencies": 8,
"max_concurrency": 256,
"image": "vllm/vllm-openai:v0.10.2"
}
],
"volume": [
{
"date": "2026-08-08",
"hardware": "h200_sxm",
"datapoints": 24
}
]
}Scroll to inspect the full example
500
The submissions query failed.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"error": {
"type": "string"
}
},
"required": [
"error"
],
"additionalProperties": true
}Response example
{
"error": "Internal server error"
}GETStable/api/v1/framework-releasesRead latest framework releases
Returns the highest-versioned non-draft, non-prerelease engine release (tag starting with v) among the 20 most recent GitHub releases for vLLM and SGLang, with its publish time. commits maps each unversioned nightly commit in an active benchmark image to its GitHub compare status against that release; behind means the commit is an ancestor of the release. Commits GitHub cannot resolve are omitted. A null value means the upstream lookup had no usable release.
Parameters
No parameters
Request
curl
curl -sS 'https://inferencex.semianalysis.com/api/v1/framework-releases'Responses
FrameworkReleases200
Framework keys mapped to the release tag, ISO 8601 publish time and nightly commit statuses, or null.
Media type: application/jsonResponse shape
{
"type": "object",
"additionalProperties": {
"oneOf": [
{
"type": "object",
"properties": {
"tag": {
"type": "string"
},
"publishedAt": {
"type": "string",
"format": "date-time"
},
"commits": {
"type": "object",
"additionalProperties": {
"type": "string",
"enum": [
"ahead",
"behind",
"diverged",
"identical"
]
}
}
},
"required": [
"tag",
"publishedAt",
"commits"
],
"additionalProperties": false
},
{
"type": "null"
}
]
}
}Scroll to inspect the full example
Response example
{
"vllm": {
"tag": "v0.10.2",
"publishedAt": "2026-08-01T12:00:00Z",
"commits": {
"af7f9488c2210d67": "behind",
"e68efc685ab1": "diverged"
}
},
"sglang": {
"tag": "v0.4.10",
"publishedAt": "2026-07-28T09:30:00Z",
"commits": {}
}
}Scroll to inspect the full example
500
The release lookup failed.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"error": {
"type": "string"
}
},
"required": [
"error"
],
"additionalProperties": true
}Response example
{
"error": "Internal server error"
}GETStable/api/v1/latest-imagesRead latest runtime images
Returns the latest container image observed for each benchmark configuration and sequence.
Parameters
No parameters
Request
curl
curl -sS 'https://inferencex.semianalysis.com/api/v1/latest-images'Responses
LatestImageRows200
Latest image rows.
Media type: application/jsonResponse shape
{
"type": "array",
"items": {
"type": "object",
"properties": {
"model": {
"type": "string"
},
"hardware": {
"type": "string"
},
"framework": {
"type": "string"
},
"precision": {
"type": "string"
},
"spec_method": {
"type": "string"
},
"disagg": {
"type": "boolean"
},
"isl": {
"type": [
"number",
"null"
]
},
"osl": {
"type": [
"number",
"null"
]
},
"benchmark_type": {
"type": "string"
},
"image": {
"type": "string"
},
"date": {
"type": "string",
"format": "date"
}
},
"required": [
"model",
"hardware",
"framework",
"precision",
"spec_method",
"disagg",
"isl",
"osl",
"benchmark_type",
"image",
"date"
],
"additionalProperties": false
}
}Scroll to inspect the full example
Response example
[
{
"model": "dsr1",
"hardware": "h200_sxm",
"framework": "vllm",
"precision": "fp8",
"spec_method": "none",
"disagg": false,
"isl": 1024,
"osl": 1024,
"benchmark_type": "single_turn",
"image": "vllm/vllm-openai:v0.10.2",
"date": "2026-08-08"
},
{
"model": "dsr1",
"hardware": "h200_sxm",
"framework": "vllm",
"precision": "fp8",
"spec_method": "none",
"disagg": false,
"isl": null,
"osl": null,
"benchmark_type": "agentic_traces",
"image": "vllm/vllm-openai:v0.10.2",
"date": "2026-08-08"
}
]Scroll to inspect the full example
500
The image query failed.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"error": {
"type": "string"
}
},
"required": [
"error"
],
"additionalProperties": true
}Response example
{
"error": "Internal server error"
}Datasets
Dataset registry, metadata, conversation indexes, and conversation structures.
GETStable/api/v1/datasetsList ingested datasets
Returns dataset registry cards without the large chart_data field.
Parameters
No parameters
Request
curl
curl -sS 'https://inferencex.semianalysis.com/api/v1/datasets'Responses
DatasetRecords200
Dataset registry records.
Media type: application/jsonResponse shape
{
"type": "array",
"items": {
"type": "object",
"properties": {
"id": {
"type": "string"
},
"slug": {
"type": "string"
},
"label": {
"type": "string"
},
"variant": {
"type": "string"
},
"description": {
"type": [
"string",
"null"
]
},
"hf_url": {
"type": [
"string",
"null"
]
},
"license": {
"type": [
"string",
"null"
]
},
"conversation_count": {
"type": "integer"
},
"summary": {
"type": "object",
"additionalProperties": true
},
"ingested_at": {
"type": "string",
"format": "date-time"
}
},
"required": [
"id",
"slug",
"label",
"variant",
"description",
"hf_url",
"license",
"conversation_count",
"summary",
"ingested_at"
],
"additionalProperties": false
}
}Scroll to inspect the full example
Response example
[
{
"id": "ds_01",
"slug": "cc-traces-weka",
"label": "CC Traces Weka",
"variant": "default",
"description": "Agentic coding traces",
"hf_url": "https://huggingface.co/datasets/example/cc-traces-weka",
"license": "Apache-2.0",
"conversation_count": 1200,
"summary": {
"totalIn": 8200000,
"totalOut": 1700000
},
"ingested_at": "2026-08-08T03:00:00Z"
}
]Scroll to inspect the full example
500
The dataset registry query failed.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"error": {
"type": "string"
}
},
"required": [
"error"
],
"additionalProperties": true
}Response example
{
"error": "Internal server error"
}GETStable/api/v1/datasets/{slug}Read dataset details
Returns one dataset registry record plus its precomputed chart_data distributions.
Parameters
| Name | Location | Type | Requirement | Description | Example |
|---|---|---|---|---|---|
| slug | path | string | Required | Dataset slug from the registry. | cc-traces-weka |
Request
curl
curl -sS 'https://inferencex.semianalysis.com/api/v1/datasets/cc-traces-weka'Responses
DatasetDetail200
Dataset metadata with chart_data.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"id": {
"type": "string"
},
"slug": {
"type": "string"
},
"label": {
"type": "string"
},
"variant": {
"type": "string"
},
"description": {
"type": [
"string",
"null"
]
},
"hf_url": {
"type": [
"string",
"null"
]
},
"license": {
"type": [
"string",
"null"
]
},
"conversation_count": {
"type": "integer"
},
"summary": {
"type": "object",
"additionalProperties": true
},
"ingested_at": {
"type": "string",
"format": "date-time"
},
"chart_data": {
"type": "object",
"additionalProperties": true
}
},
"required": [
"id",
"slug",
"label",
"variant",
"description",
"hf_url",
"license",
"conversation_count",
"summary",
"ingested_at",
"chart_data"
],
"additionalProperties": false
}Scroll to inspect the full example
Response example
{
"id": "ds_01",
"slug": "cc-traces-weka",
"label": "CC Traces Weka",
"variant": "default",
"description": "Agentic coding traces",
"hf_url": null,
"license": "Apache-2.0",
"conversation_count": 1200,
"summary": {},
"ingested_at": "2026-08-08T03:00:00Z",
"chart_data": {
"tokens": {
"bins": [
0,
1000,
2000
],
"counts": [
140,
320
]
}
}
}Scroll to inspect the full example
404
No dataset has this slug.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"error": {
"type": "string"
}
},
"required": [
"error"
],
"additionalProperties": true
}Response example
{
"error": "Not found"
}500
The dataset query failed.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"error": {
"type": "string"
}
},
"required": [
"error"
],
"additionalProperties": true
}Response example
{
"error": "Internal server error"
}GETStable/api/v1/datasets/{slug}/conversationsList dataset conversations
Returns a searchable, sorted, paginated conversation index. It contains counts only, not the full conversation structure.
Parameters
| Name | Location | Type | Requirement | Description | Example |
|---|---|---|---|---|---|
| slug | path | string | Required | Dataset slug from the registry. | cc-traces-weka |
| search | query | string | Optional | Trimmed conversation ID search, at most 100 characters. | trace-018 |
| limit | query | integer | Optional | Page size, clamped to 1 through 200. | 50 |
| offset | query | integer | Optional | Zero-based row offset. Negative values become 0. | 0 |
| sort | query | enum | Optional | Sort by tokens, turns, subagents, or id. Unknown values fall back to tokens. | tokens |
Request
curl
curl -sS 'https://inferencex.semianalysis.com/api/v1/datasets/cc-traces-weka/conversations?limit=50&offset=0&sort=tokens'Responses
ConversationList200
Total count and conversation index items.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"total": {
"type": "integer"
},
"items": {
"type": "array",
"items": {
"type": "object",
"properties": {
"conv_id": {
"type": "string"
},
"models": {
"type": "array",
"items": {
"type": "string"
}
},
"num_turns": {
"type": "integer"
},
"num_subagent_groups": {
"type": "integer"
},
"total_in": {
"type": "integer"
},
"total_out": {
"type": "integer"
},
"total_cached": {
"type": "integer"
}
},
"required": [
"conv_id",
"models",
"num_turns",
"num_subagent_groups",
"total_in",
"total_out",
"total_cached"
],
"additionalProperties": false
}
}
},
"required": [
"total",
"items"
],
"additionalProperties": false
}Scroll to inspect the full example
Response example
{
"total": 1200,
"items": [
{
"conv_id": "trace-018",
"models": [
"claude-sonnet-4"
],
"num_turns": 42,
"num_subagent_groups": 3,
"total_in": 18200,
"total_out": 4200,
"total_cached": 9600
}
]
}Scroll to inspect the full example
400
search exceeds 100 characters.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"error": {
"type": "string"
}
},
"required": [
"error"
],
"additionalProperties": true
}Response example
{
"error": "search too long"
}404
No dataset has this slug.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"error": {
"type": "string"
}
},
"required": [
"error"
],
"additionalProperties": true
}Response example
{
"error": "Not found"
}500
The conversation query failed.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"error": {
"type": "string"
}
},
"required": [
"error"
],
"additionalProperties": true
}Response example
{
"error": "Internal server error"
}GETStable/api/v1/datasets/{slug}/conversations/{convId}Read a conversation structure
Returns one conversation and its flamegraph-ready nested structure. App Router decodes each path value once.
Parameters
| Name | Location | Type | Requirement | Description | Example |
|---|---|---|---|---|---|
| slug | path | string | Required | Dataset slug from the registry. | cc-traces-weka |
| convId | path | string | Required | Conversation ID exactly as listed by the conversation index. | trace-018 |
Request
curl
curl -sS 'https://inferencex.semianalysis.com/api/v1/datasets/cc-traces-weka/conversations/trace-018'Responses
ConversationDetail200
Conversation counts and nested structure.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"conv_id": {
"type": "string"
},
"models": {
"type": "array",
"items": {
"type": "string"
}
},
"num_turns": {
"type": "integer"
},
"num_subagent_groups": {
"type": "integer"
},
"total_in": {
"type": "integer"
},
"total_out": {
"type": "integer"
},
"total_cached": {
"type": "integer"
},
"structure": {
"type": "object",
"additionalProperties": true
}
},
"required": [
"conv_id",
"models",
"num_turns",
"num_subagent_groups",
"total_in",
"total_out",
"total_cached",
"structure"
],
"additionalProperties": false
}Scroll to inspect the full example
Response example
{
"conv_id": "trace-018",
"models": [
"claude-sonnet-4"
],
"num_turns": 42,
"num_subagent_groups": 3,
"total_in": 18200,
"total_out": 4200,
"total_cached": 9600,
"structure": {
"name": "trace-018",
"children": []
}
}Scroll to inspect the full example
404
The dataset or conversation does not exist.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"error": {
"type": "string"
}
},
"required": [
"error"
],
"additionalProperties": true
}Response example
{
"error": "Not found"
}500
The conversation query failed.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"error": {
"type": "string"
}
},
"required": [
"error"
],
"additionalProperties": true
}Response example
{
"error": "Internal server error"
}CollectiveX
Versioned collective communication benchmark results, one run per results upload.
GETBeta/api/v1/collectivex/latestRead the latest CollectiveX dataset
Returns the versioned neutral dataset of the newest stored run. Each run is one uploaded CollectiveX results directory laid over the previous latest run (each uploaded case replaces the stored result of the same case) and holds at least one measured result: run_id is the upload time in epoch seconds (raised past the newest stored run when needed), run_attempt is 1, and run.source_sha is the collectivex/ commit the uploaded results were measured from. run.covered_skus lists only pools with a measured result. Runs are ingested when an operator uploads them, never on read. In series[].points[], components.*.latency_us, components.*.activation_data_rate_gbps_at_latency_percentile, components.*.payload_data_rate_gbps_at_latency_percentile and roundtrip_token_rate_at_latency_percentile carry p50, p90, p95 and p99; p90, p95 and p99 are null where the harness withheld a tail because too many samples were misaligned or host-stalled, while p50 is always a number. series[].library_version is the backend build the case ran (null where the result recorded none). Optional swap_blocks results contain verified copy latency in microseconds and payload GB/s (bytes counted once), with block_bytes, num_blocks, runtime provenance and skipped-point counts. Multi-pool sweeps preserve each GPU pool as an independent result. Each series carries maturity (candidate when no serving engine can select the configuration), and components.pair_period is the chained steady-state period, the headline latency.
Parameters
| Name | Location | Type | Requirement | Description | Example |
|---|---|---|---|---|---|
| version | query | enum | Required | CollectiveX contract version. | 1 |
Request
curl
curl -sS 'https://inferencex.semianalysis.com/api/v1/collectivex/latest?version=1'Responses
CollectiveXDataset200
Latest CollectiveX run, coverage, series, and optional KV cases.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"version": {
"type": "integer"
},
"run": {
"type": "object",
"properties": {
"run_id": {
"type": "string"
},
"run_attempt": {
"type": "integer"
},
"generated_at": {
"type": "string",
"format": "date-time"
},
"conclusion": {
"type": [
"string",
"null"
]
},
"source_sha": {
"type": "string"
},
"source_repository": {
"type": "string"
},
"requested_cases": {
"type": "integer"
},
"terminal_cases": {
"type": "integer"
},
"measured_cases": {
"type": "integer"
},
"unsupported_cases": {
"type": "integer"
},
"failed_cases": {
"type": "integer"
},
"requested_points": {
"type": "integer"
},
"terminal_points": {
"type": "integer"
},
"measured_points": {
"type": "integer"
},
"covered_skus": {
"type": "array",
"items": {
"type": "string"
}
},
"swap_requested_cases": {
"type": "integer"
},
"swap_measured_cases": {
"type": "integer"
},
"kv_requested_cases": {
"type": "integer"
},
"kv_measured_cases": {
"type": "integer"
}
},
"required": [
"run_id",
"run_attempt",
"generated_at",
"conclusion",
"source_sha",
"requested_cases",
"terminal_cases",
"measured_cases",
"unsupported_cases",
"failed_cases",
"requested_points",
"terminal_points",
"measured_points",
"covered_skus"
],
"additionalProperties": false
},
"coverage": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": true
}
},
"series": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": true
}
},
"kv": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": true
}
},
"swap_blocks": {
"type": "array",
"items": {
"type": "object",
"properties": {
"result_id": {
"type": "string"
},
"sku": {
"type": "string"
},
"runtime": {
"type": "object",
"additionalProperties": true
},
"timing": {
"type": "string"
},
"warmup": {
"type": "integer"
},
"iterations": {
"type": "integer"
},
"max_payload_bytes": {
"oneOf": [
{
"type": "integer"
},
{
"type": "null"
}
]
},
"skipped_points": {
"type": "integer"
},
"points": {
"type": "array",
"items": {
"type": "object",
"properties": {
"direction": {
"type": "string",
"enum": [
"h2d",
"d2h",
"d2d"
]
},
"layout": {
"type": "string",
"enum": [
"contiguous",
"random"
]
},
"block_bytes": {
"type": "integer"
},
"num_blocks": {
"type": "integer"
},
"payload_bytes": {
"type": "integer"
},
"seed": {
"type": "integer"
},
"host_memory": {
"type": "string"
},
"api": {
"type": "string"
},
"sample_count": {
"type": "integer"
},
"latency_us": {
"type": "object",
"additionalProperties": true
},
"payload_gbps_at_latency_percentile": {
"type": "object",
"additionalProperties": true
}
},
"required": [
"direction",
"layout",
"block_bytes",
"num_blocks",
"payload_bytes",
"seed",
"host_memory",
"api",
"sample_count",
"latency_us",
"payload_gbps_at_latency_percentile"
],
"additionalProperties": false
}
}
},
"required": [
"result_id",
"sku",
"runtime",
"timing",
"warmup",
"iterations",
"max_payload_bytes",
"skipped_points",
"points"
],
"additionalProperties": false
}
}
},
"required": [
"version",
"run",
"coverage",
"series"
],
"additionalProperties": false
}Scroll to inspect the full example
Response example
{
"version": 1,
"run": {
"run_id": "1792089351",
"run_attempt": 1,
"generated_at": "2026-10-15T18:35:51Z",
"conclusion": "success",
"source_sha": "e65c9045320c78f1fb86200802aceafa0c6310d8",
"requested_cases": 12,
"terminal_cases": 12,
"measured_cases": 10,
"unsupported_cases": 2,
"failed_cases": 0,
"requested_points": 48,
"terminal_points": 48,
"measured_points": 40,
"covered_skus": [
"h200_sxm"
],
"kv_requested_cases": 4,
"kv_measured_cases": 4
},
"coverage": [],
"series": [],
"kv": [],
"swap_blocks": []
}Scroll to inspect the full example
400
version is missing or unsupported.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"error": {
"type": "string"
}
},
"required": [
"error"
],
"additionalProperties": true
}Response example
{
"error": "Unknown version"
}404
No run is stored for the version.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"error": {
"type": "string"
}
},
"required": [
"error"
],
"additionalProperties": true
}Response example
{
"error": "Not found"
}500
The stored run query failed.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"error": {
"type": "string"
}
},
"required": [
"error"
],
"additionalProperties": true
}Response example
{
"error": "Internal server error"
}GETBeta/api/v1/collectivex/runsList CollectiveX runs
Returns every stored run summary, newest upload first. Only uploads with at least one measured result are runs, and covered_skus lists only pools with a measured result. Runs are ingested ahead of reads, so discovery_complete is always true; the field is kept for existing clients.
Parameters
| Name | Location | Type | Requirement | Description | Example |
|---|---|---|---|---|---|
| version | query | enum | Required | CollectiveX contract version. | 1 |
Request
curl
curl -sS 'https://inferencex.semianalysis.com/api/v1/collectivex/runs?version=1'Responses
CollectiveXRunList200
Version, run summaries, and discovery_complete (always true).
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"version": {
"type": "integer"
},
"runs": {
"type": "array",
"items": {
"type": "object",
"properties": {
"run_id": {
"type": "string"
},
"run_attempt": {
"type": "integer"
},
"generated_at": {
"type": "string",
"format": "date-time"
},
"conclusion": {
"type": [
"string",
"null"
]
},
"source_repository": {
"type": "string"
},
"covered_skus": {
"type": "array",
"items": {
"type": "string"
}
},
"requested_cases": {
"type": "integer"
},
"measured_cases": {
"type": "integer"
},
"requested_points": {
"type": "integer"
},
"terminal_points": {
"type": "integer"
},
"terminal_counts": {
"type": "object",
"properties": {
"measured": {
"type": "integer"
},
"unsupported": {
"type": "integer"
},
"failed": {
"type": "integer"
}
},
"required": [
"measured",
"unsupported",
"failed"
],
"additionalProperties": false
},
"swap_cases": {
"type": "object",
"properties": {
"requested": {
"type": "integer"
},
"measured": {
"type": "integer"
}
},
"required": [
"requested",
"measured"
],
"additionalProperties": false
},
"kv_cases": {
"type": "object",
"properties": {
"requested": {
"type": "integer"
},
"measured": {
"type": "integer"
}
},
"required": [
"requested",
"measured"
],
"additionalProperties": false
}
},
"required": [
"run_id",
"run_attempt",
"generated_at",
"conclusion",
"covered_skus",
"requested_cases",
"measured_cases",
"requested_points",
"terminal_points",
"terminal_counts"
],
"additionalProperties": false
}
},
"discovery_complete": {
"type": "boolean"
}
},
"required": [
"version",
"runs",
"discovery_complete"
],
"additionalProperties": false
}Scroll to inspect the full example
Response example
{
"version": 1,
"runs": [
{
"run_id": "1792089351",
"run_attempt": 1,
"generated_at": "2026-10-15T18:35:51Z",
"conclusion": "success",
"covered_skus": [
"h200_sxm"
],
"requested_cases": 12,
"measured_cases": 10,
"requested_points": 48,
"terminal_points": 48,
"terminal_counts": {
"measured": 40,
"unsupported": 8,
"failed": 0
},
"kv_cases": {
"requested": 4,
"measured": 4
}
}
],
"discovery_complete": true
}Scroll to inspect the full example
400
version is missing or unsupported.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"error": {
"type": "string"
}
},
"required": [
"error"
],
"additionalProperties": true
}Response example
{
"error": "Unknown version"
}500
The stored run list query failed.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"error": {
"type": "string"
}
},
"required": [
"error"
],
"additionalProperties": true
}Response example
{
"error": "Internal server error"
}GETBeta/api/v1/collectivex/changelogList CollectiveX changelog entries
Returns the InferenceX collectivex/perf-changelog.yaml entries linked to stored runs, newest first: one row per run and entry, with the pair-period p50 before and after for every selected case that run changed.
Parameters
| Name | Location | Type | Requirement | Description | Example |
|---|---|---|---|---|---|
| version | query | enum | Required | CollectiveX contract version. | 1 |
Request
curl
curl -sS 'https://inferencex.semianalysis.com/api/v1/collectivex/changelog?version=1'Responses
CollectiveXChangelog200
Version and linked changelog entries.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"version": {
"type": "integer"
},
"entries": {
"type": "array",
"items": {
"type": "object",
"properties": {
"run_id": {
"type": "string"
},
"entry_index": {
"type": "integer"
},
"pr_number": {
"type": "integer"
},
"pr_link": {
"type": "string"
},
"date": {
"type": "string"
},
"pools": {
"type": "array",
"items": {
"type": "string"
}
},
"selectors": {
"type": "object",
"additionalProperties": true
},
"description": {
"type": "string"
},
"changes": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": true
}
}
},
"required": [
"run_id",
"entry_index",
"pr_number",
"pr_link",
"date",
"pools",
"selectors",
"description",
"changes"
],
"additionalProperties": false
}
}
},
"required": [
"version",
"entries"
],
"additionalProperties": false
}Scroll to inspect the full example
Response example
{
"version": 1,
"entries": [
{
"run_id": "1793344844",
"entry_index": 14,
"pr_number": 4674,
"pr_link": "https://github.com/SemiAnalysisAI/InferenceX/pull/4674",
"date": "2026-10-30",
"pools": [
"mi300x"
],
"selectors": {
"backends": [
"uccl-ep"
],
"modes": [
"low-latency"
],
"precisions": [
"fp8"
]
},
"description": "Model uccl-ep's in-kernel FP8 cast exactly in low-latency mode.",
"changes": []
}
]
}Scroll to inspect the full example
400
version is missing or unsupported.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"error": {
"type": "string"
}
},
"required": [
"error"
],
"additionalProperties": true
}Response example
{
"error": "Unknown version"
}500
The changelog query failed.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"error": {
"type": "string"
}
},
"required": [
"error"
],
"additionalProperties": true
}Response example
{
"error": "Internal server error"
}GETBeta/api/v1/collectivex/runs/{runId}Read a CollectiveX run
Returns one stored run as a versioned CollectiveX dataset. The run ID is the upload time in epoch seconds (raised past the newest stored run when needed), as listed by /api/v1/collectivex/runs. In series[].points[], components.*.latency_us, components.*.activation_data_rate_gbps_at_latency_percentile, components.*.payload_data_rate_gbps_at_latency_percentile and roundtrip_token_rate_at_latency_percentile carry p50, p90, p95 and p99; p90, p95 and p99 are null where the harness withheld a tail because too many samples were misaligned or host-stalled, while p50 is always a number. Each series carries maturity (candidate when no serving engine can select the configuration), and components.pair_period is the chained steady-state period, the headline latency.
Parameters
| Name | Location | Type | Requirement | Description | Example |
|---|---|---|---|---|---|
| runId | path | integer | Required | Positive run ID: the upload time in epoch seconds, below 10^10. | 1792089351 |
| version | query | enum | Required | CollectiveX contract version. | 1 |
Request
curl
curl -sS 'https://inferencex.semianalysis.com/api/v1/collectivex/runs/1792089351?version=1'Responses
CollectiveXDataset200
The requested CollectiveX dataset.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"version": {
"type": "integer"
},
"run": {
"type": "object",
"properties": {
"run_id": {
"type": "string"
},
"run_attempt": {
"type": "integer"
},
"generated_at": {
"type": "string",
"format": "date-time"
},
"conclusion": {
"type": [
"string",
"null"
]
},
"source_sha": {
"type": "string"
},
"source_repository": {
"type": "string"
},
"requested_cases": {
"type": "integer"
},
"terminal_cases": {
"type": "integer"
},
"measured_cases": {
"type": "integer"
},
"unsupported_cases": {
"type": "integer"
},
"failed_cases": {
"type": "integer"
},
"requested_points": {
"type": "integer"
},
"terminal_points": {
"type": "integer"
},
"measured_points": {
"type": "integer"
},
"covered_skus": {
"type": "array",
"items": {
"type": "string"
}
},
"swap_requested_cases": {
"type": "integer"
},
"swap_measured_cases": {
"type": "integer"
},
"kv_requested_cases": {
"type": "integer"
},
"kv_measured_cases": {
"type": "integer"
}
},
"required": [
"run_id",
"run_attempt",
"generated_at",
"conclusion",
"source_sha",
"requested_cases",
"terminal_cases",
"measured_cases",
"unsupported_cases",
"failed_cases",
"requested_points",
"terminal_points",
"measured_points",
"covered_skus"
],
"additionalProperties": false
},
"coverage": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": true
}
},
"series": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": true
}
},
"kv": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": true
}
},
"swap_blocks": {
"type": "array",
"items": {
"type": "object",
"properties": {
"result_id": {
"type": "string"
},
"sku": {
"type": "string"
},
"runtime": {
"type": "object",
"additionalProperties": true
},
"timing": {
"type": "string"
},
"warmup": {
"type": "integer"
},
"iterations": {
"type": "integer"
},
"max_payload_bytes": {
"oneOf": [
{
"type": "integer"
},
{
"type": "null"
}
]
},
"skipped_points": {
"type": "integer"
},
"points": {
"type": "array",
"items": {
"type": "object",
"properties": {
"direction": {
"type": "string",
"enum": [
"h2d",
"d2h",
"d2d"
]
},
"layout": {
"type": "string",
"enum": [
"contiguous",
"random"
]
},
"block_bytes": {
"type": "integer"
},
"num_blocks": {
"type": "integer"
},
"payload_bytes": {
"type": "integer"
},
"seed": {
"type": "integer"
},
"host_memory": {
"type": "string"
},
"api": {
"type": "string"
},
"sample_count": {
"type": "integer"
},
"latency_us": {
"type": "object",
"additionalProperties": true
},
"payload_gbps_at_latency_percentile": {
"type": "object",
"additionalProperties": true
}
},
"required": [
"direction",
"layout",
"block_bytes",
"num_blocks",
"payload_bytes",
"seed",
"host_memory",
"api",
"sample_count",
"latency_us",
"payload_gbps_at_latency_percentile"
],
"additionalProperties": false
}
}
},
"required": [
"result_id",
"sku",
"runtime",
"timing",
"warmup",
"iterations",
"max_payload_bytes",
"skipped_points",
"points"
],
"additionalProperties": false
}
}
},
"required": [
"version",
"run",
"coverage",
"series"
],
"additionalProperties": false
}Scroll to inspect the full example
Response example
{
"version": 1,
"run": {
"run_id": "1792089351",
"run_attempt": 1,
"generated_at": "2026-10-15T18:35:51Z",
"conclusion": "success",
"source_sha": "e65c9045320c78f1fb86200802aceafa0c6310d8",
"requested_cases": 12,
"terminal_cases": 12,
"measured_cases": 10,
"unsupported_cases": 2,
"failed_cases": 0,
"requested_points": 48,
"terminal_points": 48,
"measured_points": 40,
"covered_skus": [
"h200_sxm"
],
"kv_requested_cases": 4,
"kv_measured_cases": 4
},
"coverage": [],
"series": [],
"kv": [],
"swap_blocks": []
}Scroll to inspect the full example
400
version or runId is invalid.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"error": {
"type": "string"
}
},
"required": [
"error"
],
"additionalProperties": true
}Response example
{
"error": "Unknown version or run id"
}404
The run is not stored.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"error": {
"type": "string"
}
},
"required": [
"error"
],
"additionalProperties": true
}Response example
{
"error": "Not found"
}500
The run query failed.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"error": {
"type": "string"
}
},
"required": [
"error"
],
"additionalProperties": true
}Response example
{
"error": "Internal server error"
}Diagnostic reads
Per-result trace, cache, request, sibling, and server metric diagnostics.
GETBeta/api/v1/agentic-aggregatesRead agentic aggregate percentiles
Returns ISL, OSL, KV-cache utilization, and prefix-cache hit-rate percentiles keyed by benchmark result ID. IDs are deduplicated and at most 200 are accepted.
Parameters
| Name | Location | Type | Requirement | Description | Example |
|---|---|---|---|---|---|
| ids | query | comma-separated integers | Required | One to 200 positive benchmark result IDs. | 421,422 |
Request
curl
curl -sS 'https://inferencex.semianalysis.com/api/v1/agentic-aggregates?ids=421,422'Responses
AgenticAggregateMap200
Result IDs mapped to aggregate percentiles or null metric groups.
Media type: application/jsonResponse shape
{
"type": "object",
"additionalProperties": {
"type": "object",
"properties": {
"id": {
"type": "integer"
},
"isl": {
"oneOf": [
{
"type": "object",
"properties": {
"mean": {
"type": "number"
},
"p50": {
"type": "number"
},
"p75": {
"type": "number"
},
"p90": {
"type": "number"
},
"p95": {
"type": "number"
},
"p99": {
"type": "number"
},
"n": {
"type": "integer"
}
},
"required": [
"mean",
"p50",
"p75",
"p90",
"p95",
"p99",
"n"
],
"additionalProperties": false
},
{
"type": "null"
}
]
},
"osl": {
"oneOf": [
{
"type": "object",
"properties": {
"mean": {
"type": "number"
},
"p50": {
"type": "number"
},
"p75": {
"type": "number"
},
"p90": {
"type": "number"
},
"p95": {
"type": "number"
},
"p99": {
"type": "number"
},
"n": {
"type": "integer"
}
},
"required": [
"mean",
"p50",
"p75",
"p90",
"p95",
"p99",
"n"
],
"additionalProperties": false
},
{
"type": "null"
}
]
},
"kvCacheUtil": {
"oneOf": [
{
"type": "object",
"properties": {
"mean": {
"type": "number"
},
"p50": {
"type": "number"
},
"p75": {
"type": "number"
},
"p90": {
"type": "number"
},
"p95": {
"type": "number"
},
"p99": {
"type": "number"
},
"n": {
"type": "integer"
}
},
"required": [
"mean",
"p50",
"p75",
"p90",
"p95",
"p99",
"n"
],
"additionalProperties": false
},
{
"type": "null"
}
]
},
"prefixCacheHitRate": {
"oneOf": [
{
"type": "object",
"properties": {
"mean": {
"type": "number"
},
"p50": {
"type": "number"
},
"p75": {
"type": "number"
},
"p90": {
"type": "number"
},
"p95": {
"type": "number"
},
"p99": {
"type": "number"
},
"n": {
"type": "integer"
}
},
"required": [
"mean",
"p50",
"p75",
"p90",
"p95",
"p99",
"n"
],
"additionalProperties": false
},
{
"type": "null"
}
]
}
},
"required": [
"id",
"isl",
"osl",
"kvCacheUtil",
"prefixCacheHitRate"
],
"additionalProperties": false
}
}Scroll to inspect the full example
Response example
{
"421": {
"id": 421,
"isl": {
"mean": 18320,
"p50": 16440,
"p75": 20110,
"p90": 24880,
"p95": 27940,
"p99": 31900,
"n": 512
},
"osl": null,
"kvCacheUtil": null,
"prefixCacheHitRate": null
}
}Scroll to inspect the full example
400
ids is missing, malformed, or exceeds 200 unique IDs.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"error": {
"type": "string"
}
},
"required": [
"error"
],
"additionalProperties": true
}Response example
{
"error": "ids query param is required"
}500
The aggregate query failed.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"error": {
"type": "string"
}
},
"required": [
"error"
],
"additionalProperties": true
}Response example
{
"error": "Internal server error"
}GETBeta/api/v1/benchmark-siblingsRead sibling benchmark points
Returns the benchmark SKU and every point in the same hardware, framework, model, precision, method, benchmark type, and workflow run. Siblings may have different workloads. is_current is true only for the requested result ID; false does not mean stale, invalid, or superseded.
Parameters
| Name | Location | Type | Requirement | Description | Example |
|---|---|---|---|---|---|
| id | query | integer | Required | Positive safe-integer benchmark result ID. | 421 |
Request
curl
curl -sS 'https://inferencex.semianalysis.com/api/v1/benchmark-siblings?id=421'Responses
BenchmarkSiblings200
SKU metadata and sibling navigation rows.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"sku": {
"type": "object",
"additionalProperties": true
},
"siblings": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": true
}
}
},
"required": [
"sku",
"siblings"
],
"additionalProperties": false
}Scroll to inspect the full example
Response example
{
"sku": {
"hardware": "h200_sxm",
"framework": "vllm",
"model": "dsr1",
"precision": "fp8",
"spec_method": "none",
"benchmark_type": "agentic_traces",
"github_run_id": 123456789,
"date": "2026-08-08",
"dataset_slug": "cc-traces-weka"
},
"siblings": [
{
"id": 421,
"conc": 32,
"offload_mode": "off",
"decode_tp": 8,
"decode_ep": 1,
"decode_pp": null,
"decode_dcp_size": 8,
"decode_pcp_size": 1,
"decode_dp_attention": false,
"decode_num_workers": 1,
"prefill_tp": 8,
"prefill_ep": 1,
"prefill_pp": null,
"prefill_dcp_size": 8,
"prefill_pcp_size": 1,
"prefill_dwdp_size": null,
"prefill_dp_attention": false,
"prefill_num_workers": 1,
"num_prefill_gpu": 0,
"num_decode_gpu": 8,
"disagg": false,
"is_multinode": false,
"tput_per_gpu": 128.4,
"total_requests": 320,
"is_current": true,
"has_trace": true
}
]
}Scroll to inspect the full example
400
id is missing or invalid.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"error": {
"type": "string"
}
},
"required": [
"error"
],
"additionalProperties": true
}Response example
{
"error": "id is required (benchmark_result_id)"
}404
No benchmark result has this ID.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"error": {
"type": "string"
}
},
"required": [
"error"
],
"additionalProperties": true
}Response example
{
"error": "Not found"
}500
The sibling query failed.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"error": {
"type": "string"
}
},
"required": [
"error"
],
"additionalProperties": true
}Response example
{
"error": "Internal server error"
}GETBeta/api/v1/derived-agentic-metricsRead derived agentic metrics
Returns normalized interactivity percentiles keyed by benchmark result ID. IDs are deduplicated and at most 200 are accepted.
Parameters
| Name | Location | Type | Requirement | Description | Example |
|---|---|---|---|---|---|
| ids | query | comma-separated integers | Required | One to 200 positive benchmark result IDs. | 421,422 |
Request
curl
curl -sS 'https://inferencex.semianalysis.com/api/v1/derived-agentic-metrics?ids=421,422'Responses
DerivedAgenticMetricMap200
Result IDs mapped to p75 and p90 normalized interactivity.
Media type: application/jsonResponse shape
{
"type": "object",
"additionalProperties": {
"type": "object",
"properties": {
"id": {
"type": "integer"
},
"p75_e2e_norm_intvty": {
"type": [
"number",
"null"
]
},
"p90_e2e_norm_intvty": {
"type": [
"number",
"null"
]
}
},
"required": [
"id",
"p75_e2e_norm_intvty",
"p90_e2e_norm_intvty"
],
"additionalProperties": false
}
}Scroll to inspect the full example
Response example
{
"421": {
"id": 421,
"p75_e2e_norm_intvty": 31.2,
"p90_e2e_norm_intvty": 24.8
}
}400
ids is missing, malformed, or exceeds 200 unique IDs.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"error": {
"type": "string"
}
},
"required": [
"error"
],
"additionalProperties": true
}Response example
{
"error": "ids query param is required"
}500
The derived metric query failed.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"error": {
"type": "string"
}
},
"required": [
"error"
],
"additionalProperties": true
}Response example
{
"error": "Internal server error"
}GETBeta/api/v1/request-timelineRead a request timeline
Returns a versioned benchmark window and per-request replay identity, source provenance, dispatch, acknowledgement, completion, token, phase, worker, and cancellation timing. Current stored timelines stream directly as gzip JSON with all request records preserved (decompressed for clients whose Accept-Encoding excludes gzip, and without an ETag). Once a timeline has been copied to the content-addressed CDN store, clients that accept gzip get a 307 redirect to those exact gzip bytes, served as Content-Type application/gzip without Content-Encoding, so decompress the body yourself (for example curl -L ... | gunzip). Legacy timelines retain the same response shape.
Parameters
| Name | Location | Type | Requirement | Description | Example |
|---|---|---|---|---|---|
| id | query | integer | Required | Positive safe-integer benchmark result ID. | 421 |
Request
curl
curl -sS 'https://inferencex.semianalysis.com/api/v1/request-timeline?id=421'Responses
RequestTimeline200
Timeline metadata and request records. Nanosecond event fields are offsets from startNs.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"version": {
"type": "integer"
},
"startNs": {
"type": "integer"
},
"endNs": {
"type": "integer"
},
"durationS": {
"type": "number"
},
"requests": {
"type": "array",
"items": {
"type": "object",
"properties": {
"cid": {
"type": "string"
},
"ri": {
"type": "integer"
},
"ti": {
"type": "integer"
},
"srcTrace": {
"type": "string"
},
"srcOuter": {
"type": "integer"
},
"srcInner": {
"type": "integer"
},
"srcKind": {
"type": "string"
},
"wid": {
"type": "string"
},
"ad": {
"type": "integer"
},
"phase": {
"type": "string"
},
"credit": {
"type": "integer"
},
"start": {
"type": "integer"
},
"ack": {
"type": [
"number",
"null"
]
},
"end": {
"type": "integer"
},
"ttftMs": {
"type": [
"number",
"null"
]
},
"tpotMs": {
"type": [
"number",
"null"
]
},
"isl": {
"type": [
"number",
"null"
]
},
"osl": {
"type": [
"number",
"null"
]
},
"cancelled": {
"type": "boolean"
}
},
"required": [
"cid",
"ti",
"wid",
"ad",
"phase",
"credit",
"start",
"ack",
"end",
"ttftMs",
"tpotMs",
"isl",
"osl",
"cancelled"
],
"additionalProperties": false
}
}
},
"required": [
"version",
"startNs",
"endNs",
"durationS",
"requests"
],
"additionalProperties": false
}Scroll to inspect the full example
Response example
{
"version": 6,
"startNs": 1000000000,
"endNs": 2400000000,
"durationS": 1.4,
"requests": [
{
"cid": "trace-018",
"ri": 0,
"ti": 0,
"wid": "7",
"ad": 0,
"phase": "profiling",
"credit": 0,
"start": 1200000,
"ack": 1800000,
"end": 420000000,
"ttftMs": 42.3,
"tpotMs": 18.1,
"isl": 18320,
"osl": 410,
"cancelled": false
}
]
}Scroll to inspect the full example
400
id is missing or invalid.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"error": {
"type": "string"
}
},
"required": [
"error"
],
"additionalProperties": true
}Response example
{
"error": "id is required (benchmark_result_id)"
}404
No timeline exists for this result.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"error": {
"type": "string"
}
},
"required": [
"error"
],
"additionalProperties": true
}Response example
{
"error": "Not found"
}500
The timeline query failed.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"error": {
"type": "string"
}
},
"required": [
"error"
],
"additionalProperties": true
}Response example
{
"error": "Internal server error"
}GETBeta/api/v1/server-logRead a benchmark server log
Returns one stored .log/.out file for a benchmark result ID. Use file with a name from server-log-files. Add offset or limit for a bounded chunk; add download=1 to stream the complete selected file as a text attachment. Without offset, limit, or download, files over 16 MiB return 413; download=1 returns 413 for files over 512 MiB, which must be read in offset/limit ranges. Stored logs never change, so successful responses are CDN-cached for a year.
Parameters
| Name | Location | Type | Requirement | Description | Example |
|---|---|---|---|---|---|
| id | query | integer | Required | Positive benchmark result ID. | 421 |
| file | query | string | Optional | Artifact-relative filename. Omit it to read the primary/legacy file. | results/router.log |
| offset | query | integer | Optional | Zero-based character offset. Supplying offset or limit enables chunked mode. | 0 |
| limit | query | integer | Optional | Chunk size in characters, from 1 to 262144. Defaults to 65536. | 65536 |
| download | query | integer | Optional | Set to 1 to stream the complete selected file as a text attachment. Cannot be combined with offset or limit. | 1 |
Request
curl
curl -sS 'https://inferencex.semianalysis.com/api/v1/server-log?id=421&file=results%2Frouter.log&offset=0&limit=65536'Responses
ServerLog | ServerLogChunk | text/plain attachment200
Benchmark result ID and server log text, plus range metadata in chunked mode.
Media type: application/jsonResponse shape
{
"oneOf": [
{
"type": "object",
"properties": {
"id": {
"type": "integer"
},
"serverLog": {
"type": "string"
}
},
"required": [
"id",
"serverLog"
],
"additionalProperties": false
},
{
"type": "object",
"properties": {
"id": {
"type": "integer"
},
"fileName": {
"type": "string"
},
"serverLog": {
"type": "string"
},
"offset": {
"type": "integer"
},
"nextOffset": {
"type": [
"number",
"null"
]
}
},
"required": [
"id",
"fileName",
"serverLog",
"offset",
"nextOffset"
],
"additionalProperties": false
}
]
}Scroll to inspect the full example
Response example
{
"id": 421,
"fileName": "results/router.log",
"serverLog": "INFO router initialized\n",
"offset": 0,
"nextOffset": 24
}400
id is missing or invalid.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"error": {
"type": "string"
}
},
"required": [
"error"
],
"additionalProperties": true
}Response example
{
"error": "id is required (benchmark_result_id)"
}404
No server log exists for this result.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"error": {
"type": "string"
}
},
"required": [
"error"
],
"additionalProperties": true
}Response example
{
"error": "Not found"
}413
The file exceeds the size limit for a complete read (16 MiB, JSON body with bytes and maxBytes) or a download (512 MiB, plain-text body).
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"error": {
"type": "string"
}
},
"required": [
"error"
],
"additionalProperties": true
}Response example
{
"error": "Log file is 951652286 bytes, above the 16777216-byte limit for this mode. Read it with offset and limit, or stream it with download=1."
}500
The server log query failed.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"error": {
"type": "string"
}
},
"required": [
"error"
],
"additionalProperties": true
}Response example
{
"error": "Internal server error"
}GETBeta/api/v1/server-log-searchSearch complete benchmark logs
Runs a literal, case-insensitive search across every stored .log/.out file for one benchmark result. The bounded response contains contextual snippets without transferring complete log files. One request scans at most 64 million characters across all files; incomplete is true when that budget stopped the scan before the end. A search that exceeds its time budget returns 504.
Parameters
| Name | Location | Type | Requirement | Description | Example |
|---|---|---|---|---|---|
| id | query | integer | Required | Positive benchmark result ID. | 421 |
| q | query | string | Required | Literal search text, from 1 to 256 characters. | router ready |
| limit | query | integer | Optional | Maximum matches to return, from 1 to 100. Defaults to 50. | 50 |
Request
curl
curl -sS 'https://inferencex.semianalysis.com/api/v1/server-log-search?id=421&q=router%20ready&limit=50'Responses
ServerLogSearchResult200
Bounded matches with filenames, character offsets, and contextual text.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"id": {
"type": "integer"
},
"query": {
"type": "string"
},
"matches": {
"type": "array",
"items": {
"type": "object",
"properties": {
"fileName": {
"type": "string"
},
"offset": {
"type": "integer"
},
"before": {
"type": "string"
},
"match": {
"type": "string"
},
"after": {
"type": "string"
}
},
"required": [
"fileName",
"offset",
"before",
"match",
"after"
],
"additionalProperties": false
}
},
"truncated": {
"type": "boolean"
},
"incomplete": {
"type": "boolean"
}
},
"required": [
"id",
"query",
"matches",
"truncated",
"incomplete"
],
"additionalProperties": false
}Scroll to inspect the full example
Response example
{
"id": 421,
"query": "router ready",
"matches": [
{
"fileName": "results/router.log",
"offset": 128,
"before": "INFO ",
"match": "router ready",
"after": " on port 8000\n"
}
],
"truncated": false,
"incomplete": false
}Scroll to inspect the full example
400
The search parameters are invalid.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"error": {
"type": "string"
}
},
"required": [
"error"
],
"additionalProperties": true
}Response example
{
"error": "Invalid query"
}504
The search exceeded its time budget.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"error": {
"type": "string"
}
},
"required": [
"error"
],
"additionalProperties": true
}Response example
{
"error": "Search timed out. Try a more specific term."
}500
The complete-log search failed.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"error": {
"type": "string"
}
},
"required": [
"error"
],
"additionalProperties": true
}Response example
{
"error": "Internal server error"
}GETBeta/api/v1/server-log-filesList stored benchmark log files
Lists every .log and .out filename retained from the matching server-log artifact. Paths are returned relative to the artifact root, with the primary file first.
Parameters
| Name | Location | Type | Requirement | Description | Example |
|---|---|---|---|---|---|
| id | query | integer | Required | Positive safe-integer benchmark result ID. | 421 |
Request
curl
curl -sS 'https://inferencex.semianalysis.com/api/v1/server-log-files?id=421'Responses
ServerLogFileNames200
Artifact-relative .log/.out filenames.
Media type: application/jsonResponse shape
{
"type": "array",
"items": {
"type": "string"
}
}Response example
[
"results/server.log",
"results/benchmark.log",
"results/router.log"
]400
id is missing or invalid.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"error": {
"type": "string"
}
},
"required": [
"error"
],
"additionalProperties": true
}Response example
{
"error": "id is required (benchmark_result_id)"
}404
No log bundle exists for this result.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"error": {
"type": "string"
}
},
"required": [
"error"
],
"additionalProperties": true
}Response example
{
"error": "Not found"
}500
The log filename query failed.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"error": {
"type": "string"
}
},
"required": [
"error"
],
"additionalProperties": true
}Response example
{
"error": "Internal server error"
}GETBeta/api/v1/log-availabilityCheck server-log availability
Returns only benchmark result IDs that have a stored server log. IDs are deduplicated and at most 500 are accepted.
Parameters
| Name | Location | Type | Requirement | Description | Example |
|---|---|---|---|---|---|
| ids | query | comma-separated integers | Required | One to 500 positive benchmark result IDs. | 421,422 |
Request
curl
curl -sS 'https://inferencex.semianalysis.com/api/v1/log-availability?ids=421,422'Responses
LogAvailabilityMap200
Available result IDs mapped to true. Missing keys have no server log.
Media type: application/jsonResponse shape
{
"type": "object",
"additionalProperties": {
"type": "boolean"
}
}Response example
{
"421": true
}400
ids is missing, malformed, or exceeds 500 unique IDs.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"error": {
"type": "string"
}
},
"required": [
"error"
],
"additionalProperties": true
}Response example
{
"error": "Expected ids as comma-separated positive integers"
}500
The server-log availability query failed.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"error": {
"type": "string"
}
},
"required": [
"error"
],
"additionalProperties": true
}Response example
{
"error": "Internal server error"
}GETBeta/api/v1/trace-availabilityCheck trace availability
Returns only benchmark result IDs that have a stored trace. IDs are deduplicated and at most 500 are accepted.
Parameters
| Name | Location | Type | Requirement | Description | Example |
|---|---|---|---|---|---|
| ids | query | comma-separated integers | Required | One to 500 positive benchmark result IDs. | 421,422 |
Request
curl
curl -sS 'https://inferencex.semianalysis.com/api/v1/trace-availability?ids=421,422'Responses
TraceAvailabilityMap200
Available result IDs mapped to true. Missing keys have no trace.
Media type: application/jsonResponse shape
{
"type": "object",
"additionalProperties": {
"type": "boolean"
}
}Response example
{
"421": true
}400
ids is missing, malformed, or exceeds 500 unique IDs.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"error": {
"type": "string"
}
},
"required": [
"error"
],
"additionalProperties": true
}Response example
{
"error": "ids query param is required"
}500
The trace availability query failed.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"error": {
"type": "string"
}
},
"required": [
"error"
],
"additionalProperties": true
}Response example
{
"error": "Internal server error"
}GETBeta/api/v1/gpu-metrics-point/statsRead power load-swing statistics
Load-swing statistics of the PowerX power trace recorded while one benchmark point ran, from the stored one-second buckets of the per-GPU mean: peak, min, idle floor (P5 before the serving window), window mean, P5/P95, largest 1 s and 5 s change, ramp up/down per second and volatility (std ÷ mean), with the recorded serving window and the count of telemetry gaps that were skipped rather than bridged. One entry per linked power series (a single-node artifact, or one validation window of a sweep bundle). The same function serves the PowerX Over time view. scale=nvl72 multiplies by 72 into kW per rack as an upper bound that assumes every GPU swings together. format=csv returns the per-second trace instead. Every read checks the live telemetry revision; responses are no-store.
Parameters
| Name | Location | Type | Requirement | Description | Example |
|---|---|---|---|---|---|
| id | query | integer | Required | Positive safe-integer benchmark result ID. | 445805 |
| format | query | enum | Optional | json returns the statistics; csv returns the per-second trace (artifact, source, t_s, since_serving_s, in_validated_window, w_per_gpu or kw_per_nvl72_rack). | json |
| scale | query | enum | Optional | gpu: W per GPU (the per-GPU mean of each bucket). nvl72: × 72 in kW per NVL72 rack, an upper bound that assumes all GPUs swing together; volatility is unchanged. | gpu |
Request
curl
curl -sS 'https://inferencex.semianalysis.com/api/v1/gpu-metrics-point/stats?id=445805'Responses
GpuMetricsPointLoadSwingStats200
Statistics per linked power series, with definitions and caveats.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"benchmarkResultId": {
"type": "integer",
"minimum": 1,
"maximum": 9007199254740991
},
"apiVersion": {
"type": "string",
"enum": [
"v1"
]
},
"scale": {
"type": "string",
"enum": [
"gpu",
"nvl72"
]
},
"unit": {
"type": "string",
"enum": [
"W per GPU",
"kW per NVL72 rack"
]
},
"gpusPerRack": {
"type": "integer"
},
"definitions": {
"type": "object",
"additionalProperties": {
"type": "string"
}
},
"caveats": {
"type": "array",
"items": {
"type": "string"
}
},
"series": {
"type": "array",
"items": {
"type": "object",
"properties": {
"artifact": {
"type": "string"
},
"source": {
"type": [
"string",
"null"
]
},
"gpuCount": {
"type": "integer"
},
"window": {
"oneOf": [
{
"type": "object",
"properties": {
"startSeconds": {
"type": "number"
},
"endSeconds": {
"type": "number"
}
},
"required": [
"startSeconds",
"endSeconds"
],
"additionalProperties": false
},
{
"type": "null"
}
]
},
"stats": {
"type": "object",
"properties": {
"unit": {
"type": "string"
},
"sampleIntervalSeconds": {
"type": "number"
},
"sampleCount": {
"type": "integer"
},
"windowSampleCount": {
"type": "integer"
},
"preServingSampleCount": {
"type": "integer"
},
"gapCount": {
"type": "integer"
},
"peak": {
"type": "number"
},
"min": {
"type": "number"
},
"idleFloor": {
"type": [
"number",
"null"
]
},
"mean": {
"type": [
"number",
"null"
]
},
"p5": {
"type": [
"number",
"null"
]
},
"p95": {
"type": [
"number",
"null"
]
},
"maxDelta1s": {
"type": "number"
},
"maxDelta5s": {
"type": "number"
},
"rampUpPerSecond": {
"type": "number"
},
"rampDownPerSecond": {
"type": "number"
},
"volatility": {
"type": [
"number",
"null"
]
}
},
"required": [
"unit",
"sampleIntervalSeconds",
"sampleCount",
"windowSampleCount",
"preServingSampleCount",
"gapCount",
"peak",
"min",
"idleFloor",
"mean",
"p5",
"p95",
"maxDelta1s",
"maxDelta5s",
"rampUpPerSecond",
"rampDownPerSecond",
"volatility"
],
"additionalProperties": false
}
},
"required": [
"artifact",
"source",
"gpuCount",
"window",
"stats"
],
"additionalProperties": false
}
}
},
"required": [
"benchmarkResultId",
"apiVersion",
"scale",
"unit",
"gpusPerRack",
"definitions",
"caveats",
"series"
],
"additionalProperties": false
}Scroll to inspect the full example
Response example
{
"benchmarkResultId": 445805,
"apiVersion": "v1",
"scale": "gpu",
"unit": "W per GPU",
"gpusPerRack": 72,
"definitions": {
"peak_min_delta_ramp": "whole run, per-GPU mean of each one-second bucket",
"mean_p5_p95_volatility": "validated serving window only",
"idle_floor": "P5 of the samples before the serving window starts",
"ramp": "largest change over a 5-second span divided by that span",
"gaps": "buckets more than 1.5× the sample interval apart are counted and never bridged"
},
"caveats": [
"One-second DCGM samples; sub-second swings are not captured.",
"Single node or NVL72 rack, inference only.",
"The NVL72 scale multiplies the per-GPU trace by 72 and assumes every GPU swings together, so it is an upper bound on rack-level swings."
],
"series": [
{
"artifact": "gpu_metrics_dsv4pro_8k1k_fp4_vllm_tp8_c64",
"source": null,
"gpuCount": 8,
"window": {
"startSeconds": 212,
"endSeconds": 1407
},
"stats": {
"unit": "W per GPU",
"sampleIntervalSeconds": 1,
"sampleCount": 1480,
"windowSampleCount": 1196,
"preServingSampleCount": 212,
"gapCount": 0,
"peak": 1012.4,
"min": 141.2,
"idleFloor": 143.8,
"mean": 871.5,
"p5": 744.1,
"p95": 968.3,
"maxDelta1s": 318.6,
"maxDelta5s": 612.9,
"rampUpPerSecond": 122.6,
"rampDownPerSecond": 97.4,
"volatility": 0.071
}
}
]
}Scroll to inspect the full example
Alternate representation: text/csv
Response shape
{
"type": "string"
}Response example
artifact,source,t_s,since_serving_s,in_validated_window,w_per_gpu
gpu_metrics_dsv4pro_8k1k_fp4_vllm_tp8_c64,,0,-212,0,143.8400
id is missing or not a positive safe integer, or format / scale is not an allowed value.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"error": {
"type": "string"
}
},
"required": [
"error"
],
"additionalProperties": true
}Response example
{
"error": "id is required (benchmark_result_id)"
}404
The point has no linked telemetry or no usable power series.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"error": {
"type": "string"
}
},
"required": [
"error"
],
"additionalProperties": true
}Response example
{
"error": "Not found"
}500
The telemetry lookup failed.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"error": {
"type": "string"
}
},
"required": [
"error"
],
"additionalProperties": true
}Response example
{
"error": "Internal server error"
}GETBeta/api/v1/trace-histogramsRead trace histograms
Returns input and output token count arrays for each benchmark result ID. IDs are deduplicated and at most 200 are accepted. Once a response has been copied to the content-addressed CDN store, clients that accept gzip get a 307 redirect to the gzip of the same JSON, served as Content-Type application/gzip without Content-Encoding, so decompress the body yourself.
Parameters
| Name | Location | Type | Requirement | Description | Example |
|---|---|---|---|---|---|
| ids | query | comma-separated integers | Required | One to 200 positive benchmark result IDs. | 421,422 |
Request
curl
curl -sS 'https://inferencex.semianalysis.com/api/v1/trace-histograms?ids=421,422'Responses
TraceHistogramMap200
Result IDs mapped to raw ISL and OSL samples.
Media type: application/jsonResponse shape
{
"type": "object",
"additionalProperties": {
"type": "object",
"properties": {
"id": {
"type": "integer"
},
"isl": {
"type": "array",
"items": {
"type": "number"
}
},
"osl": {
"type": "array",
"items": {
"type": "number"
}
}
},
"required": [
"id",
"isl",
"osl"
],
"additionalProperties": false
}
}Scroll to inspect the full example
Response example
{
"421": {
"id": 421,
"isl": [
18220,
19340,
15110
],
"osl": [
410,
380,
512
]
}
}Scroll to inspect the full example
400
ids is missing, malformed, or exceeds 200 unique IDs.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"error": {
"type": "string"
}
},
"required": [
"error"
],
"additionalProperties": true
}Response example
{
"error": "ids query param is required"
}500
The histogram query failed.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"error": {
"type": "string"
}
},
"required": [
"error"
],
"additionalProperties": true
}Response example
{
"error": "Internal server error"
}GETBeta/api/v1/trace-server-metricsRead trace server metrics
Returns point metadata, including prefill/decode TP, EP, DP attention, worker counts, physical GPU counts and recipe_fingerprint for cache-reuse links, and chart-ready aggregate time series for cache usage, queue depth, prefill and decode throughput, and prompt-token sources. metricSources contains source descriptors; source-specific arrays are loaded by the point-detail UI only when selected. TokenSpeed exports device KV utilization and running/waiting request gauges; unavailable cache-hit and token-throughput measurements remain empty.
Parameters
| Name | Location | Type | Requirement | Description | Example |
|---|---|---|---|---|---|
| id | query | integer | Required | Positive safe-integer benchmark result ID. | 421 |
Request
curl
curl -sS 'https://inferencex.semianalysis.com/api/v1/trace-server-metrics?id=421'Responses
TraceServerMetrics200
Point metadata, window bounds, and server metric series.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"meta": {
"type": "object",
"additionalProperties": true,
"properties": {
"prefill_tp": {
"type": "integer"
},
"prefill_ep": {
"type": "integer"
},
"prefill_dp_attention": {
"type": "boolean"
},
"prefill_num_workers": {
"type": "integer"
},
"decode_tp": {
"type": "integer"
},
"decode_ep": {
"type": "integer"
},
"decode_dp_attention": {
"type": "boolean"
},
"decode_num_workers": {
"type": "integer"
},
"num_prefill_gpu": {
"type": "integer"
},
"num_decode_gpu": {
"type": "integer"
},
"recipe_fingerprint": {
"type": [
"string",
"null"
]
}
}
},
"startNs": {
"type": "integer"
},
"endNs": {
"type": "integer"
},
"durationS": {
"type": "number"
},
"timeslicesCount": {
"type": "integer"
},
"kvCacheUsage": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": true
}
},
"prefixCacheHitRate": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": true
}
},
"queueDepth": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": true
}
},
"promptTokensBySource": {
"type": "object",
"additionalProperties": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": true
}
}
},
"prefillTps": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": true
}
},
"decodeTps": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": true
}
},
"prefixCacheHitsTps": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": true
}
},
"hostKvCacheUsage": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": true
}
},
"kvCacheUsageByEngine": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": true
}
},
"kvCachePoolTokens": {
"type": [
"number",
"null"
]
},
"metricSources": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": true
}
}
},
"required": [
"meta",
"startNs",
"endNs",
"durationS",
"timeslicesCount",
"kvCacheUsage",
"prefixCacheHitRate",
"queueDepth",
"promptTokensBySource",
"prefillTps",
"decodeTps",
"prefixCacheHitsTps",
"hostKvCacheUsage",
"kvCacheUsageByEngine",
"kvCachePoolTokens",
"metricSources"
],
"additionalProperties": false
}Scroll to inspect the full example
Response example
{
"meta": {
"id": 421,
"hardware": "h200_sxm",
"framework": "vllm",
"model": "dsr1",
"conc": 32,
"date": "2026-08-08"
},
"startNs": 1000000000,
"endNs": 2400000000,
"durationS": 1.4,
"timeslicesCount": 2,
"kvCacheUsage": [
{
"t": 0,
"v": 0.44
}
],
"prefixCacheHitRate": [],
"queueDepth": [],
"promptTokensBySource": {},
"prefillTps": [],
"decodeTps": [],
"prefixCacheHitsTps": [],
"hostKvCacheUsage": [],
"kvCacheUsageByEngine": [],
"kvCachePoolTokens": 983040,
"metricSources": []
}Scroll to inspect the full example
400
id is missing or invalid.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"error": {
"type": "string"
}
},
"required": [
"error"
],
"additionalProperties": true
}Response example
{
"error": "id is required (benchmark_result_id)"
}404
No server metrics exist for this result.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"error": {
"type": "string"
}
},
"required": [
"error"
],
"additionalProperties": true
}Response example
{
"error": "Not found"
}500
The server metric query failed.
Media type: application/jsonResponse shape
{
"type": "object",
"properties": {
"error": {
"type": "string"
}
},
"required": [
"error"
],
"additionalProperties": true
}Response example
{
"error": "Internal server error"
}