中文

Agentic inference costs

Hyperscaler cost ↓ Lower is better

Source: InferenceX & SemiAnalysis Market July 2026 Pricing Surveys & AI Cloud TCO Model

Benchmark config

  • Muse Spark 1.3/Open 816B

    Long-context multi-turn realistic agentic scenario (AgentX)

    B200
    $0.012
    SGLang · FP4
    MI355X
    $0.013
    SGLang · FP4
    B300
    $0.013
    SGLang · FP4
    GB200 NVL72
    —no exact @100 result
    GB300 NVL72
    $0.010
    Dynamo SGLang · FP4

  • DeepSeek V4.1 Flash 552B

    Long-context multi-turn realistic agentic scenario (AgentX)

    B200
    —no exact @100 result
    MI355X
    $0.00340% cheaper than this platform’s Oct 12 result
    vLLM · FP4
    Compare curves
    B300
    $0.00156% cheaper than this platform’s Oct 2 result
    vLLM · FP4
    Compare curves
    GB200 NVL72
    —no exact @100 result
    GB300 NVL72
    $0.00156% cheaper than this platform’s Oct 2 result
    vLLM · FP4
    Compare curves

  • Kimi K3.1 Pro 2.8T

    Long-context multi-turn realistic agentic scenario (AgentX)

    B200
    —cannot reach @100
    MI355X
    $0.075
    vLLM · FP4
    B300
    $0.100
    SGLang · FP4
    GB200 NVL72
    $0.154
    Dynamo SGLang · FP4
    GB300 NVL72
    $0.050
    Dynamo vLLM · FP4

  • MiniMax M3.1 Flash 428B

    Long-context multi-turn realistic agentic scenario (AgentX)

    B200
    $0.005
    SGLang · FP4
    MI355X
    $0.005
    SGLang · FP4
    B300
    $0.005
    SGLang · FP4
    GB200 NVL72
    $0.009
    Dynamo vLLM · FP4
    GB300 NVL72
    $0.005
    SGLang · FP4

  • GLM5.2/GLM5.3 744B

    Long-context multi-turn realistic agentic scenario (AgentX)

    B200
    $0.02953% cheaper than this platform’s Oct 2 result
    Dynamo SGLang · FP4
    Compare curves
    MI355X
    $0.0287% more expensive than this platform’s Oct 6 result
    SGLang · FP4
    Compare curves
    B300
    $0.03021% cheaper than this platform’s Oct 12 result
    Dynamo SGLang · FP4
    Compare curves
    GB200 NVL72
    —no exact @100 result
    GB300 NVL72
    $0.026
    Dynamo SGLang · FP4

  • Qwen3.5 397B

    Long-context multi-turn realistic agentic scenario (AgentX)

    B200
    $0.00744% cheaper than this platform’s Sep 16 result
    SGLang · FP4
    Compare curves
    MI355X
    $0.007
    SGLang · FP4
    B300
    $0.00772% cheaper than this platform’s Sep 21 result
    Dynamo SGLang · FP4
    Compare curves
    GB200 NVL72
    $0.007
    Dynamo SGLang · FP4
    GB300 NVL72
    $0.012
    Dynamo SGLang · FP4

  • Qwen3.8 Flash Next 176B

    Long-context multi-turn realistic agentic scenario (AgentX)

    B200
    —no exact @100 result
    MI355X
    $0.008
    SGLang · FP4
    B300
    $0.006
    SGLang · FP4
    GB200 NVL72
    —no data for this scenario
    GB300 NVL72
    $0.00457% cheaper than this platform’s Oct 9 result
    SGLang · FP4
    Compare curves

  • Qwen3.5 397B

    8K/1K

    B200
    $0.04438% cheaper than this platform’s Sep 18 result
    SGLang · FP4
    Compare curves
    MI355X
    $0.075
    SGLang · FP4
    B300
    $0.054
    SGLang · FP4
    GB200 NVL72
    $0.06981% cheaper than this platform’s Sep 18 result
    Dynamo SGLang · FP4
    Compare curves
    GB300 NVL72
    $0.133
    Dynamo SGLang · FP8

Current cost and change versus the latest validated platform result 30–60 days earlier.

Platforms without a valid 30-day comparison show current cost only.

If a chip does not have FP4 spec decoding available, the next best available configuration is used.