All model and GPU pairings

Muse Spark 1.3/Open 816BNVIDIA Hopper

Running Muse Spark 1.3/Open on H200

Quick answer

Muse Spark 1.3/Open sustains 9,323 tokens/s per GPU on H200 at 50 tokens/s per user, which works out to $0.036 per million tokens at hyperscaler pricing, served by SGLang. Fastest measured TTFT: 0.7 ms; fastest TPOT: 0.0 ms (each the best across all configs, not one run).

Benchmarked configs
7
Serving engines
SGLang
Precisions
FP4
Run dates
2026-10-19

Throughput at every interactivity target

Serving is a trade-off: push more concurrent users through a GPU and each user's tokens arrive slower. The ladder below reads the measured frontier at each per-user speed target on the AgentX agentic coding workload, using the best engine and precision at that point.

Per-user targetTokens/s per GPU$ / 1M tokensEnginePrecision
30 tok/s--SGLangfp4
50 tok/s9,323$0.036SGLangfp4
75 tok/s6,786$0.050SGLangfp4
100 tok/s5,519$0.061SGLangfp4
150 tok/s2,677$0.13SGLangfp4
200 tok/s--SGLangfp4

What serving actually costs

Converting the 50 tokens/s per user operating point to $ per million total tokens across rental pricing tiers from the SemiAnalysis AI Cloud TCO model.

Pricing tier$ / GPU / hr$ / 1M tokens
Owning at large hyperscaler volume$1.22$0.036
Retail$2.90$0.086

Frequently asked questions

How fast is Muse Spark 1.3/Open on H200?
At an interactivity target of 50 tokens/s per user on the AgentX agentic coding workload, H200 sustains 9,323 tokens/s per GPU serving Muse Spark 1.3/Open with SGLang in FP4. Peak measured throughput across all configs is 11,387 tokens/s per GPU.
How much does it cost to serve Muse Spark 1.3/Open on H200?
$0.036 per million total tokens at large-hyperscaler-volume ownership $/GPU/hr pricing, at 50 tokens/s per user. The retail rental tier is tabulated above; slower interactivity targets lower the cost further.
Which serving engines run Muse Spark 1.3/Open on H200?
The runs behind this page used SGLang in FP4. Engines are rebuilt and re-benchmarked continuously, so the best config can change between visits.
How are these Muse Spark 1.3/Open numbers measured?
Every number is measured on real H200 hardware by the InferenceX fleet, sweeping concurrency on the AgentX agentic coding workload to trace the throughput-versus-interactivity frontier; the newest run landed on 2026-10-19. The same derivation powers the InferenceX overview leaderboard.

Explore the data