Inference Cost per Million Tokens

Hyperscaler cost ↓ Lower is better Source: InferenceX & SemiAnalysis Market July 2026 AI Cloud TCO Model

Current cost and change versus the latest validated platform result 30–60 days earlier.
Model · Scenario	B200	MI355X	B300	GB200 NVL72	GB300 NVL72
DeepSeek V4 Pro 1.6T 8K/1K	$0.304 vLLM · FP4	$0.35567% cheaper than this platform’s Jul 4 result SGLang · FP4 Compare curves	$0.245 SGLang · FP4	$0.514 Dynamo vLLM · FP4	$0.21349% cheaper than this platform’s Jul 6 result Dynamo SGLang · FP4 · STP Compare curves
DeepSeek V4 Pro 1.6T Long Context Multi-Turn Realistic Agentic Scenario (AgentX)	$0.050 vLLM · FP4	$0.256 MoRI SGLang · FP4	$0.034 vLLM · FP4	no data for this scenario	$0.011 Dynamo vLLM · FP4
Kimi K3 2.8T Long Context Multi-Turn Realistic Agentic Scenario (AgentX)	$1.350 Dynamo vLLM · FP4 · STP	no data for this scenario	$0.154 vLLM · FP4	$0.497 Dynamo vLLM · FP4	no data for this scenario
MiniMax M3 428B 8K/1K	$0.06713% cheaper than this platform’s Jun 23 result vLLM · FP8 Compare curves	$0.113 vLLM · FP4	$0.05612% cheaper than this platform’s Jul 6 result vLLM · FP4 Compare curves	$0.10814% more expensive than this platform’s Jun 29 result Dynamo vLLM · FP8 · STP Compare curves	$0.110About the same cost as this platform’s Jun 26 result Dynamo vLLM · FP8 · STP Compare curves
MiniMax M3 428B Long Context Multi-Turn Realistic Agentic Scenario (AgentX)	no exact @50 result	no data for this scenario	$0.012 vLLM · FP4	no data for this scenario	no data for this scenario
GLM5.2 Long Context Multi-Turn Realistic Agentic Scenario (AgentX)	no exact @50 result	no data for this scenario	no exact @50 result	no data for this scenario	no data for this scenario
Qwen3.5 397B 8K/1K	$0.040About the same cost as this platform’s Jul 5 result SGLang · FP4 Compare curves	$0.06727% cheaper than this platform’s Jun 15 result SGLang · FP4 Compare curves	$0.0406% cheaper than this platform’s Jul 4 result SGLang · FP4 Compare curves	$0.142 Dynamo SGLang · FP8 · STP	$0.13341% more expensive than this platform’s Jun 25 result Dynamo SGLang · FP8 · STP Compare curves
Qwen3.5 397B Long Context Multi-Turn Realistic Agentic Scenario (AgentX)	no exact @50 result	no data for this scenario	$0.006 SGLang · FP4	no data for this scenario	$0.007 Dynamo SGLang · FP4
DeepSeek R1 0528 671BMaintenance 8K/1K	$0.10414% cheaper than this platform’s Jul 2 result SGLang · FP4 Compare curves	$0.259 SGLang · FP8	$0.353 SGLang · FP8	$0.192 Dynamo SGLang · FP8 · STP	$0.263 Dynamo SGLang · FP8 · STP
Kimi K2.5/2.6/2.7-Code 1TDeprecated 8K/1K	$0.139 Dynamo vLLM · FP4 · STP	$0.14745% cheaper than this platform’s Jul 5 result vLLM · FP4 · STP Compare curves	$0.179 vLLM · FP4 · STP	$0.112 Dynamo vLLM · FP4 · STP	$0.115 Dynamo vLLM · FP4 · STP
GLM5/5.1 744BDeprecated 8K/1K	$0.172 SGLang · FP4	$0.431 SGLang · FP8	$0.210 SGLang · FP4	$0.333 Dynamo SGLang · FP4 · STP	$0.433 Dynamo SGLang · FP4 · STP
gpt-oss 120BDeprecated 8K/1K	$0.011 vLLM · FP4 · STP	$0.011 vLLM · FP4 · STP	no data for this scenario	no data for this scenario	no data for this scenario
MiniMax M2.5/2.7 230BDeprecated 8K/1K	$0.041 vLLM · FP4 · STP	$0.054 vLLM · FP4 · STP	$0.051 vLLM · FP4 · STP	$0.041 Dynamo vLLM · FP4 · STP	$0.047 Dynamo vLLM · FP4 · STP
Llama 3.3 70B InstructDeprecated 8K/1K	$0.078 vLLM · FP4 · STP	$0.145 vLLM · FP4 · STP	no data for this scenario	no data for this scenario	no data for this scenario

DeepSeek V4 Pro 1.6T
8K/1K
B200
$0.304
vLLM · FP4
MI355X
$0.35567% cheaper than this platform’s Jul 4 result
SGLang · FP4
Compare curves
B300
$0.245
SGLang · FP4
GB200 NVL72
$0.514
Dynamo vLLM · FP4
GB300 NVL72
$0.21349% cheaper than this platform’s Jul 6 result
Dynamo SGLang · FP4STP
Compare curves
DeepSeek V4 Pro 1.6T
Long Context Multi-Turn Realistic Agentic Scenario (AgentX)
B200
$0.050
vLLM · FP4
MI355X
$0.256
MoRI SGLang · FP4
B300
$0.034
vLLM · FP4
GB200 NVL72
no data for this scenario
GB300 NVL72
$0.011
Dynamo vLLM · FP4
Kimi K3 2.8T
Long Context Multi-Turn Realistic Agentic Scenario (AgentX)
B200
$1.350
Dynamo vLLM · FP4STP
MI355X
no data for this scenario
B300
$0.154
vLLM · FP4
GB200 NVL72
$0.497
Dynamo vLLM · FP4
GB300 NVL72
no data for this scenario
MiniMax M3 428B
8K/1K
B200
$0.06713% cheaper than this platform’s Jun 23 result
vLLM · FP8
Compare curves
MI355X
$0.113
vLLM · FP4
B300
$0.05612% cheaper than this platform’s Jul 6 result
vLLM · FP4
Compare curves
GB200 NVL72
$0.10814% more expensive than this platform’s Jun 29 result
Dynamo vLLM · FP8STP
Compare curves
GB300 NVL72
$0.110About the same cost as this platform’s Jun 26 result
Dynamo vLLM · FP8STP
Compare curves
MiniMax M3 428B
Long Context Multi-Turn Realistic Agentic Scenario (AgentX)
B200
no exact @50 result
MI355X
no data for this scenario
B300
$0.012
vLLM · FP4
GB200 NVL72
no data for this scenario
GB300 NVL72
no data for this scenario
GLM5.2
Long Context Multi-Turn Realistic Agentic Scenario (AgentX)
B200
no exact @50 result
MI355X
no data for this scenario
B300
no exact @50 result
GB200 NVL72
no data for this scenario
GB300 NVL72
no data for this scenario
Qwen3.5 397B
8K/1K
B200
$0.040About the same cost as this platform’s Jul 5 result
SGLang · FP4
Compare curves
MI355X
$0.06727% cheaper than this platform’s Jun 15 result
SGLang · FP4
Compare curves
B300
$0.0406% cheaper than this platform’s Jul 4 result
SGLang · FP4
Compare curves
GB200 NVL72
$0.142
Dynamo SGLang · FP8STP
GB300 NVL72
$0.13341% more expensive than this platform’s Jun 25 result
Dynamo SGLang · FP8STP
Compare curves
Qwen3.5 397B
Long Context Multi-Turn Realistic Agentic Scenario (AgentX)
B200
no exact @50 result
MI355X
no data for this scenario
B300
$0.006
SGLang · FP4
GB200 NVL72
no data for this scenario
GB300 NVL72
$0.007
Dynamo SGLang · FP4
DeepSeek R1 0528 671BMaintenance
8K/1K
B200
$0.10414% cheaper than this platform’s Jul 2 result
SGLang · FP4
Compare curves
MI355X
$0.259
SGLang · FP8
B300
$0.353
SGLang · FP8
GB200 NVL72
$0.192
Dynamo SGLang · FP8STP
GB300 NVL72
$0.263
Dynamo SGLang · FP8STP
Kimi K2.5/2.6/2.7-Code 1TDeprecated
8K/1K
B200
$0.139
Dynamo vLLM · FP4STP
MI355X
$0.14745% cheaper than this platform’s Jul 5 result
vLLM · FP4STP
Compare curves
B300
$0.179
vLLM · FP4STP
GB200 NVL72
$0.112
Dynamo vLLM · FP4STP
GB300 NVL72
$0.115
Dynamo vLLM · FP4STP
GLM5/5.1 744BDeprecated
8K/1K
B200
$0.172
SGLang · FP4
MI355X
$0.431
SGLang · FP8
B300
$0.210
SGLang · FP4
GB200 NVL72
$0.333
Dynamo SGLang · FP4STP
GB300 NVL72
$0.433
Dynamo SGLang · FP4STP
gpt-oss 120BDeprecated
8K/1K
B200
$0.011
vLLM · FP4STP
MI355X
$0.011
vLLM · FP4STP
B300
no data for this scenario
GB200 NVL72
no data for this scenario
GB300 NVL72
no data for this scenario
MiniMax M2.5/2.7 230BDeprecated
8K/1K
B200
$0.041
vLLM · FP4STP
MI355X
$0.054
vLLM · FP4STP
B300
$0.051
vLLM · FP4STP
GB200 NVL72
$0.041
Dynamo vLLM · FP4STP
GB300 NVL72
$0.047
Dynamo vLLM · FP4STP
Llama 3.3 70B InstructDeprecated
8K/1K
B200
$0.078
vLLM · FP4STP
MI355X
$0.145
vLLM · FP4STP
B300
no data for this scenario
GB200 NVL72
no data for this scenario
GB300 NVL72
no data for this scenario

Current cost and change versus the latest validated platform result 30–60 days earlier.

Platforms without a valid 30-day comparison show current cost only.

If a chip does not have FP4 spec decoding available, the next best available configuration is used.

DeepSeek V4 Pro 1.6T

DeepSeek V4 Pro 1.6T

Kimi K3 2.8T

MiniMax M3 428B

MiniMax M3 428B

GLM5.2

Qwen3.5 397B

Qwen3.5 397B

DeepSeek R1 0528 671BMaintenance

Kimi K2.5/2.6/2.7-Code 1TDeprecated

GLM5/5.1 744BDeprecated

gpt-oss 120BDeprecated

MiniMax M2.5/2.7 230BDeprecated

Llama 3.3 70B InstructDeprecated

DeepSeek V4 Pro 1.6T

DeepSeek V4 Pro 1.6T

Kimi K3 2.8T

MiniMax M3 428B

MiniMax M3 428B

GLM5.2

Qwen3.5 397B

Qwen3.5 397B

DeepSeek R1 0528 671BMaintenance

Kimi K2.5/2.6/2.7-Code 1TDeprecated

GLM5/5.1 744BDeprecated

gpt-oss 120BDeprecated

MiniMax M2.5/2.7 230BDeprecated

Llama 3.3 70B InstructDeprecated