Agentic Inference Costs

Hyperscaler cost ↓ Lower is better Source: InferenceX & SemiAnalysis Market July 2026 Pricing Surveys & AI Cloud TCO Model

Current cost and change versus the latest validated platform result 30–60 days earlier.
Model · Scenario	B200	MI355X	B300	GB200 NVL72	GB300 NVL72
DeepSeek V4 Pro 0813 1.6T Long Context Multi-Turn Realistic Agentic Scenario (AgentX)	$0.00777% cheaper than this platform’s Aug 19 result vLLM · FP4 Compare curves	$0.01863% cheaper than this platform’s Aug 14 result SGLang · FP4 Compare curves	$0.01142% cheaper than this platform’s Aug 11 result SGLang · FP4 Compare curves	— no exact @50 result	$0.011About the same cost as this platform’s Aug 14 result Dynamo vLLM · FP4 Compare curves
DeepSeek V4.1 Flash 552B Long Context Multi-Turn Realistic Agentic Scenario (AgentX)	$0.055 SGLang · FP4	$0.011 vLLM · FP4	$0.004 vLLM · FP4	$0.006 vLLM · FP4	$0.007 vLLM · FP4
Kimi K3 2.8T Long Context Multi-Turn Realistic Agentic Scenario (AgentX)	$0.10514% cheaper than this platform’s Aug 19 result Dynamo vLLM · FP4 Compare curves	$0.06375% cheaper than this platform’s Aug 12 result vLLM · FP4 Compare curves	$0.103 vLLM · FP4	$0.11431% cheaper than this platform’s Aug 18 result Dynamo vLLM · FP4 Compare curves	$0.03852% cheaper than this platform’s Aug 18 result Dynamo vLLM · FP4 Compare curves
MiniMax M3 428B Long Context Multi-Turn Realistic Agentic Scenario (AgentX)	— no exact @50 result	— no exact @50 result	— no exact @50 result	— no exact @50 result	— no exact @50 result
GLM5.2/GLM5.3 744B Long Context Multi-Turn Realistic Agentic Scenario (AgentX)	— no exact @50 result	$0.03026% cheaper than this platform’s Aug 11 result SGLang · FP4 Compare curves	— no exact @50 result	— no exact @50 result	— no exact @50 result
Qwen3.5 397B Long Context Multi-Turn Realistic Agentic Scenario (AgentX)	$0.018 SGLang · FP8	$0.00683% cheaper than this platform’s Aug 20 result SGLang · FP4 Compare curves	$0.014 SGLang · FP8	$0.006 Dynamo SGLang · FP4	— no exact @50 result
Qwen3.8 Flash Next 176B Long Context Multi-Turn Realistic Agentic Scenario (AgentX)	— no exact @50 result	— no data for this scenario	— no exact @50 result	— no data for this scenario	— no data for this scenario
Qwen3.5 397B 8K/1K	$0.04928% more expensive than this platform’s Aug 13 result SGLang · FP8 Compare curves	$0.05124% cheaper than this platform’s Aug 6 result SGLang · FP4 Compare curves	$0.0368% cheaper than this platform’s Aug 10 result SGLang · FP4 Compare curves	— no exact @50 result	— no exact @50 result
DeepSeek R1 0528 671BMaintenanceModel is no longer actively benchmarked. 8K/1K	$0.104 SGLang · FP4	$0.259 SGLang · FP8	$0.353 SGLang · FP8	$0.192 Dynamo SGLang · FP8 · STP	$0.263 Dynamo SGLang · FP8 · STP
Kimi K2.5/2.6/2.7-Code 1TDeprecatedModel is no longer actively benchmarked. 8K/1K	$0.139 Dynamo vLLM · FP4 · STP	$0.147 vLLM · FP4 · STP	$0.179 vLLM · FP4 · STP	$0.112 Dynamo vLLM · FP4 · STP	$0.115 Dynamo vLLM · FP4 · STP
GLM5/5.1 744BDeprecatedModel is no longer actively benchmarked. 8K/1K	$0.172 SGLang · FP4	$0.431 SGLang · FP8	$0.210 SGLang · FP4	$0.333 Dynamo SGLang · FP4 · STP	$0.433 Dynamo SGLang · FP4 · STP
gpt-oss 120BDeprecatedModel is no longer actively benchmarked. 8K/1K	$0.011 vLLM · FP4 · STP	$0.011 vLLM · FP4 · STP	— no data for this scenario	— no data for this scenario	— no data for this scenario
MiniMax M2.5/2.7 230BDeprecatedModel is no longer actively benchmarked. 8K/1K	$0.041 vLLM · FP4 · STP	$0.054 vLLM · FP4 · STP	$0.051 vLLM · FP4 · STP	$0.041 Dynamo vLLM · FP4 · STP	$0.047 Dynamo vLLM · FP4 · STP
Llama 3.3 70B InstructDeprecatedModel is no longer actively benchmarked. 8K/1K	$0.078 vLLM · FP4 · STP	$0.145 vLLM · FP4 · STP	— no data for this scenario	— no data for this scenario	— no data for this scenario

DeepSeek V4 Pro 0813 1.6T
Long Context Multi-Turn Realistic Agentic Scenario (AgentX)
B200
$0.00777% cheaper than this platform’s Aug 19 result
vLLM · FP4
Compare curves
MI355X
$0.01863% cheaper than this platform’s Aug 14 result
SGLang · FP4
Compare curves
B300
$0.01142% cheaper than this platform’s Aug 11 result
SGLang · FP4
Compare curves
GB200 NVL72
—
no exact @50 result
GB300 NVL72
$0.011About the same cost as this platform’s Aug 14 result
Dynamo vLLM · FP4
Compare curves
DeepSeek V4.1 Flash 552B
Long Context Multi-Turn Realistic Agentic Scenario (AgentX)
B200
$0.055
SGLang · FP4
MI355X
$0.011
vLLM · FP4
B300
$0.004
vLLM · FP4
GB200 NVL72
$0.006
vLLM · FP4
GB300 NVL72
$0.007
vLLM · FP4
Kimi K3 2.8T
Long Context Multi-Turn Realistic Agentic Scenario (AgentX)
B200
$0.10514% cheaper than this platform’s Aug 19 result
Dynamo vLLM · FP4
Compare curves
MI355X
$0.06375% cheaper than this platform’s Aug 12 result
vLLM · FP4
Compare curves
B300
$0.103
vLLM · FP4
GB200 NVL72
$0.11431% cheaper than this platform’s Aug 18 result
Dynamo vLLM · FP4
Compare curves
GB300 NVL72
$0.03852% cheaper than this platform’s Aug 18 result
Dynamo vLLM · FP4
Compare curves
MiniMax M3 428B
Long Context Multi-Turn Realistic Agentic Scenario (AgentX)
B200
—
no exact @50 result
MI355X
—
no exact @50 result
B300
—
no exact @50 result
GB200 NVL72
—
no exact @50 result
GB300 NVL72
—
no exact @50 result
GLM5.2/GLM5.3 744B
Long Context Multi-Turn Realistic Agentic Scenario (AgentX)
B200
—
no exact @50 result
MI355X
$0.03026% cheaper than this platform’s Aug 11 result
SGLang · FP4
Compare curves
B300
—
no exact @50 result
GB200 NVL72
—
no exact @50 result
GB300 NVL72
—
no exact @50 result
Qwen3.5 397B
Long Context Multi-Turn Realistic Agentic Scenario (AgentX)
B200
$0.018
SGLang · FP8
MI355X
$0.00683% cheaper than this platform’s Aug 20 result
SGLang · FP4
Compare curves
B300
$0.014
SGLang · FP8
GB200 NVL72
$0.006
Dynamo SGLang · FP4
GB300 NVL72
—
no exact @50 result
Qwen3.8 Flash Next 176B
Long Context Multi-Turn Realistic Agentic Scenario (AgentX)
B200
—
no exact @50 result
MI355X
—
no data for this scenario
B300
—
no exact @50 result
GB200 NVL72
—
no data for this scenario
GB300 NVL72
—
no data for this scenario
Qwen3.5 397B
8K/1K
B200
$0.04928% more expensive than this platform’s Aug 13 result
SGLang · FP8
Compare curves
MI355X
$0.05124% cheaper than this platform’s Aug 6 result
SGLang · FP4
Compare curves
B300
$0.0368% cheaper than this platform’s Aug 10 result
SGLang · FP4
Compare curves
GB200 NVL72
—
no exact @50 result
GB300 NVL72
—
no exact @50 result
DeepSeek R1 0528 671BMaintenanceModel is no longer actively benchmarked.
8K/1K
B200
$0.104
SGLang · FP4
MI355X
$0.259
SGLang · FP8
B300
$0.353
SGLang · FP8
GB200 NVL72
$0.192
Dynamo SGLang · FP8STP
GB300 NVL72
$0.263
Dynamo SGLang · FP8STP
Kimi K2.5/2.6/2.7-Code 1TDeprecatedModel is no longer actively benchmarked.
8K/1K
B200
$0.139
Dynamo vLLM · FP4STP
MI355X
$0.147
vLLM · FP4STP
B300
$0.179
vLLM · FP4STP
GB200 NVL72
$0.112
Dynamo vLLM · FP4STP
GB300 NVL72
$0.115
Dynamo vLLM · FP4STP
GLM5/5.1 744BDeprecatedModel is no longer actively benchmarked.
8K/1K
B200
$0.172
SGLang · FP4
MI355X
$0.431
SGLang · FP8
B300
$0.210
SGLang · FP4
GB200 NVL72
$0.333
Dynamo SGLang · FP4STP
GB300 NVL72
$0.433
Dynamo SGLang · FP4STP
gpt-oss 120BDeprecatedModel is no longer actively benchmarked.
8K/1K
B200
$0.011
vLLM · FP4STP
MI355X
$0.011
vLLM · FP4STP
B300
—
no data for this scenario
GB200 NVL72
—
no data for this scenario
GB300 NVL72
—
no data for this scenario
MiniMax M2.5/2.7 230BDeprecatedModel is no longer actively benchmarked.
8K/1K
B200
$0.041
vLLM · FP4STP
MI355X
$0.054
vLLM · FP4STP
B300
$0.051
vLLM · FP4STP
GB200 NVL72
$0.041
Dynamo vLLM · FP4STP
GB300 NVL72
$0.047
Dynamo vLLM · FP4STP
Llama 3.3 70B InstructDeprecatedModel is no longer actively benchmarked.
8K/1K
B200
$0.078
vLLM · FP4STP
MI355X
$0.145
vLLM · FP4STP
B300
—
no data for this scenario
GB200 NVL72
—
no data for this scenario
GB300 NVL72
—
no data for this scenario

Current cost and change versus the latest validated platform result 30–60 days earlier.

Platforms without a valid 30-day comparison show current cost only.

If a chip does not have FP4 spec decoding available, the next best available configuration is used.

DeepSeek V4 Pro 0813 1.6T

DeepSeek V4.1 Flash 552B

Kimi K3 2.8T

MiniMax M3 428B

GLM5.2/GLM5.3 744B

Qwen3.5 397B

Qwen3.8 Flash Next 176B

Qwen3.5 397B

DeepSeek R1 0528 671BMaintenanceModel is no longer actively benchmarked.

Kimi K2.5/2.6/2.7-Code 1TDeprecatedModel is no longer actively benchmarked.

GLM5/5.1 744BDeprecatedModel is no longer actively benchmarked.

gpt-oss 120BDeprecatedModel is no longer actively benchmarked.

MiniMax M2.5/2.7 230BDeprecatedModel is no longer actively benchmarked.

Llama 3.3 70B InstructDeprecatedModel is no longer actively benchmarked.

DeepSeek V4 Pro 0813 1.6T

DeepSeek V4.1 Flash 552B

Kimi K3 2.8T

MiniMax M3 428B

GLM5.2/GLM5.3 744B

Qwen3.5 397B

Qwen3.8 Flash Next 176B

Qwen3.5 397B

DeepSeek R1 0528 671BMaintenanceModel is no longer actively benchmarked.

Kimi K2.5/2.6/2.7-Code 1TDeprecatedModel is no longer actively benchmarked.

GLM5/5.1 744BDeprecatedModel is no longer actively benchmarked.

gpt-oss 120BDeprecatedModel is no longer actively benchmarked.

MiniMax M2.5/2.7 230BDeprecatedModel is no longer actively benchmarked.

Llama 3.3 70B InstructDeprecatedModel is no longer actively benchmarked.