Agentic Inference Costs

Hyperscaler cost ↓ Lower is better

Source: InferenceX & SemiAnalysis Market July 2026 Pricing Surveys & AI Cloud TCO Model

Official API prices: each model developer’s own pricing page

Benchmark Config
  • Muse Spark 1.3/Open 816B

    Long-Context Multi-Turn Realistic Agentic Scenario (AgentX)

    B200
    $0.01236% cheaper than this platform’s Oct 19 result
    SGLang · FP4
    B200 self-hosted $0.012 vs official API $0.192 per 1M total tokens on this workload: ×16 cheaper than API.Compare curves
    MI355X
    $0.00985% cheaper than this platform’s Oct 19 result
    SGLang · FP4
    MI355X self-hosted $0.009 vs official API $0.192 per 1M total tokens on this workload: ×20 cheaper than API.Compare curves
    B300
    $0.01336% cheaper than this platform’s Oct 19 result
    SGLang · FP4
    B300 self-hosted $0.013 vs official API $0.192 per 1M total tokens on this workload: ×15 cheaper than API.Compare curves
    GB200 NVL72
    —no exact @100 result
    GB300 NVL72
    $0.01057% cheaper than this platform’s Oct 19 result
    Dynamo SGLang · FP4
    GB300 NVL72 self-hosted $0.010 vs official API $0.192 per 1M total tokens on this workload: ×19 cheaper than API.Compare curves
    Official API
    $0.192
    Meta
  • DeepSeek V4.1 Flash 552B

    Long-Context Multi-Turn Realistic Agentic Scenario (AgentX)

    B200
    —no exact @100 result
    MI355X
    $0.00321% cheaper than this platform’s Oct 13 result
    SGLang · FP4
    MI355X self-hosted $0.003 vs official API $0.009 per 1M total tokens on this workload: ×3.5 cheaper than API.Compare curves
    B300
    $0.00160% cheaper than this platform’s Oct 16 result
    Dynamo vLLM · FP4
    B300 self-hosted $0.001 vs official API $0.009 per 1M total tokens on this workload: ×11 cheaper than API.Compare curves
    GB200 NVL72
    —no exact @100 result
    GB300 NVL72
    $0.00173% cheaper than this platform’s Oct 14 result
    Dynamo vLLM · FP4
    GB300 NVL72 self-hosted $0.001 vs official API $0.009 per 1M total tokens on this workload: ×14 cheaper than API.Compare curves
    Official API
    $0.009
    DeepSeek
  • Kimi K3.1 Pro 2.8T

    Long-Context Multi-Turn Realistic Agentic Scenario (AgentX)

    B200
    Dynamo SGLang · FP4
    B200 self-hosted $0.162 vs official API $0.284 per 1M total tokens on this workload: ×1.8 cheaper than API.
    MI355X
    SGLang · FP4
    MI355X self-hosted $0.072 vs official API $0.284 per 1M total tokens on this workload: ×3.9 cheaper than API.
    B300
    Dynamo vLLM · FP4
    B300 self-hosted $0.028 vs official API $0.284 per 1M total tokens on this workload: ×10.0 cheaper than API.
    GB200 NVL72
    Dynamo SGLang · FP4
    GB200 NVL72 self-hosted $0.056 vs official API $0.284 per 1M total tokens on this workload: ×5.1 cheaper than API.
    GB300 NVL72
    Dynamo vLLM · FP4
    GB300 NVL72 self-hosted $0.024 vs official API $0.284 per 1M total tokens on this workload: ×12 cheaper than API.
    Official API
    $0.284
    Moonshot AI
  • MiniMax M3.1 Flash 428B

    Long-Context Multi-Turn Realistic Agentic Scenario (AgentX)

    B200
    SGLang · FP4
    B200 self-hosted $0.005 vs official API $0.021 per 1M total tokens on this workload: ×3.8 cheaper than API.
    MI355X
    SGLang · FP4
    MI355X self-hosted $0.005 vs official API $0.021 per 1M total tokens on this workload: ×3.8 cheaper than API.
    B300
    SGLang · FP4
    B300 self-hosted $0.005 vs official API $0.021 per 1M total tokens on this workload: ×4.5 cheaper than API.
    GB200 NVL72
    Dynamo vLLM · FP4
    GB200 NVL72 self-hosted $0.009 vs official API $0.021 per 1M total tokens on this workload: ×2.3 cheaper than API.
    GB300 NVL72
    SGLang · FP4
    GB300 NVL72 self-hosted $0.005 vs official API $0.021 per 1M total tokens on this workload: ×4.0 cheaper than API.
    Official API
    $0.021
    MiniMax
  • GLM5.2/GLM5.3 744B

    Long-Context Multi-Turn Realistic Agentic Scenario (AgentX)

    B200
    $0.02717% cheaper than this platform’s Oct 19 result
    Dynamo SGLang · FP4
    B200 self-hosted $0.027 vs official API $0.303 per 1M total tokens on this workload: ×11 cheaper than API.Compare curves
    MI355X
    $0.0285% more expensive than this platform’s Oct 6 result
    SGLang · FP4
    MI355X self-hosted $0.028 vs official API $0.303 per 1M total tokens on this workload: ×11 cheaper than API.Compare curves
    B300
    Dynamo SGLang · FP4
    B300 self-hosted $0.030 vs official API $0.303 per 1M total tokens on this workload: ×10.0 cheaper than API.
    GB200 NVL72
    $0.01457% cheaper than this platform’s Oct 7 result
    Dynamo SGLang · FP4
    GB200 NVL72 self-hosted $0.014 vs official API $0.303 per 1M total tokens on this workload: ×22 cheaper than API.Compare curves
    GB300 NVL72
    $0.01350% cheaper than this platform’s Oct 8 result
    Dynamo SGLang · FP4
    GB300 NVL72 self-hosted $0.013 vs official API $0.303 per 1M total tokens on this workload: ×24 cheaper than API.Compare curves
    Official API
    $0.303
    Z.ai
  • Qwen3.5 397B

    Long-Context Multi-Turn Realistic Agentic Scenario (AgentX)

    B200
    SGLang · FP4
    B200 self-hosted $0.007 vs official API $0.334 per 1M total tokens on this workload: ×45 cheaper than API.
    MI355X
    SGLang · FP4
    MI355X self-hosted $0.007 vs official API $0.334 per 1M total tokens on this workload: ×48 cheaper than API.
    B300
    $0.0078% cheaper than this platform’s Oct 14 result
    SGLang · FP4
    B300 self-hosted $0.007 vs official API $0.334 per 1M total tokens on this workload: ×48 cheaper than API.Compare curves
    GB200 NVL72
    Dynamo SGLang · FP4
    GB200 NVL72 self-hosted $0.007 vs official API $0.334 per 1M total tokens on this workload: ×45 cheaper than API.
    GB300 NVL72
    Dynamo SGLang · FP4
    GB300 NVL72 self-hosted $0.006 vs official API $0.334 per 1M total tokens on this workload: ×60 cheaper than API.
    Official API
    $0.334
    Alibaba Cloud
  • Qwen3.8 Flash Next 176B

    Long-Context Multi-Turn Realistic Agentic Scenario (AgentX)

    B200
    —no exact @100 result
    MI355X
    SGLang · FP4
    MI355X self-hosted $0.008 vs official API $0.022 per 1M total tokens on this workload: ×2.6 cheaper than API.
    B300
    SGLang · FP4
    B300 self-hosted $0.006 vs official API $0.022 per 1M total tokens on this workload: ×3.7 cheaper than API.
    GB200 NVL72
    —no data for this scenario
    GB300 NVL72
    $0.00457% cheaper than this platform’s Oct 9 result
    SGLang · FP4
    GB300 NVL72 self-hosted $0.004 vs official API $0.022 per 1M total tokens on this workload: ×5.2 cheaper than API.Compare curves
    Official API
    $0.022
    Alibaba Cloud
  • Qwen3.5 397B

    8K/1K

    B200
    SGLang · FP4
    B200 self-hosted $0.044 vs official API $0.268 per 1M total tokens on this workload: ×6.1 cheaper than API.
    MI355X
    SGLang · FP4
    MI355X self-hosted $0.075 vs official API $0.268 per 1M total tokens on this workload: ×3.6 cheaper than API.
    B300
    SGLang · FP4
    B300 self-hosted $0.054 vs official API $0.268 per 1M total tokens on this workload: ×5.0 cheaper than API.
    GB200 NVL72
    Dynamo SGLang · FP4
    GB200 NVL72 self-hosted $0.041 vs official API $0.268 per 1M total tokens on this workload: ×6.5 cheaper than API.
    GB300 NVL72
    Dynamo SGLang · FP8
    GB300 NVL72 self-hosted $0.069 vs official API $0.268 per 1M total tokens on this workload: ×3.9 cheaper than API.
    Official API
    $0.268
    Alibaba Cloud

Current cost and change versus the latest validated platform result 30–60 days earlier.

Platforms without a valid 30-day comparison show current cost only.

If a chip does not have FP4 spec decoding available, the next best available configuration is used.

Official API: the model developer’s own list price, never a reseller’s, converted with the workload’s cached-input / uncached-input / output token mix. It does not change with the SLO, engine scope, comparison or reference hardware; “Hide unchanged” and “Hide blanks” only remove whole rows, judged by their hardware results. The corpus counts every reusable prefix as a cache hit, so on cached APIs the figure is a lower bound.