Inference Cost per Million Tokens

Hyperscaler cost ↓ Lower is better Source: InferenceX & SemiAnalysis Market July 2026 AI Cloud TCO Model

Current cost and change versus the latest validated platform result 30–60 days earlier.
Model · Scenario	B200	MI355X	B300	GB200 NVL72	GB300 NVL72
DeepSeek V4 Pro 1.6T 8K/1K	$0.112 TRTLLM · FP4	$0.15985% cheaper than this platform’s Jul 4 result ATOM¹ · FP4 Compare curves	$0.134 TRTLLM · FP4	$0.514 Dynamo vLLM · FP4	$0.21317% cheaper than this platform’s Jun 15 result Dynamo SGLang · FP4 · STP Compare curves
DeepSeek V4 Pro 1.6T Long Context Multi-Turn Realistic Agentic Scenario (AgentX)	$0.050 vLLM · FP4	$0.256 MoRI SGLang · FP4	$0.034 vLLM · FP4	— no data for this scenario	$0.011 Dynamo vLLM · FP4
Kimi K3 2.8T Long Context Multi-Turn Realistic Agentic Scenario (AgentX)	$1.350 Dynamo vLLM · FP4 · STP	— no data for this scenario	$0.154 vLLM · FP4	$0.497 Dynamo vLLM · FP4	— no data for this scenario
MiniMax M3 428B 8K/1K	$0.06713% cheaper than this platform’s Jun 23 result vLLM · FP8 Compare curves	$0.053 ATOM¹ · FP4	$0.05612% cheaper than this platform’s Jul 6 result vLLM · FP4 Compare curves	$0.10814% more expensive than this platform’s Jun 29 result Dynamo vLLM · FP8 · STP Compare curves	$0.110About the same cost as this platform’s Jun 26 result Dynamo vLLM · FP8 · STP Compare curves
MiniMax M3 428B Long Context Multi-Turn Realistic Agentic Scenario (AgentX)	— no exact @50 result	— no data for this scenario	$0.012 vLLM · FP4	— no data for this scenario	— no data for this scenario
GLM5.2 Long Context Multi-Turn Realistic Agentic Scenario (AgentX)	— no exact @50 result	— no data for this scenario	— no exact @50 result	— no data for this scenario	— no data for this scenario
Qwen3.5 397B 8K/1K	$0.036 TRTLLM · FP4	$0.06727% cheaper than this platform’s Jun 15 result SGLang · FP4 Compare curves	$0.0406% cheaper than this platform’s Jul 4 result SGLang · FP4 Compare curves	$0.142 Dynamo SGLang · FP8 · STP	$0.06333% cheaper than this platform’s Jun 25 result Dynamo TRTLLM · FP4 · STP Compare curves
Qwen3.5 397B Long Context Multi-Turn Realistic Agentic Scenario (AgentX)	— no exact @50 result	— no data for this scenario	$0.006 SGLang · FP4	— no data for this scenario	$0.007 Dynamo SGLang · FP4
DeepSeek R1 0528 671BMaintenance 8K/1K	$0.077 TRTLLM · FP4	$0.157 ATOM¹ · FP4	$0.139 Dynamo TRTLLM · FP4	$0.109 Dynamo TRTLLM · FP4 · STP	$0.230 Dynamo TRTLLM · FP8 · STP
Kimi K2.5/2.6/2.7-Code 1TDeprecated 8K/1K	$0.084 Dynamo TRTLLM · FP4 · STP	$0.132 ATOM¹ · FP4 · STP	$0.179 vLLM · FP4 · STP	$0.079 Dynamo TRTLLM · FP4 · STP	$0.098 Dynamo TRTLLM · FP4 · STP
GLM5/5.1 744BDeprecated 8K/1K	$0.172 SGLang · FP4	$0.431 SGLang · FP8	$0.210 SGLang · FP4	$0.26022% cheaper than this platform’s Jul 8 result Dynamo TRTLLM · FP4 · STP Compare curves	$0.281 Dynamo TRTLLM · FP4 · STP
gpt-oss 120BDeprecated 8K/1K	$0.010 TRTLLM · FP4 · STP	$0.010 ATOM¹ · FP4 · STP	— no data for this scenario	— no exact @50 result	— no data for this scenario
MiniMax M2.5/2.7 230BDeprecated 8K/1K	$0.041 vLLM · FP4 · STP	$0.051 ATOM¹ · FP4 · STP	$0.051 vLLM · FP4 · STP	$0.041 Dynamo vLLM · FP4 · STP	$0.047 Dynamo vLLM · FP4 · STP
Llama 3.3 70B InstructDeprecated 8K/1K	$0.065 TRTLLM · FP4 · STP	$0.145 vLLM · FP4 · STP	— no data for this scenario	— no data for this scenario	— no data for this scenario

DeepSeek V4 Pro 1.6T
8K/1K
B200
$0.112
TRTLLM · FP4
MI355X
$0.15985% cheaper than this platform’s Jul 4 result
ATOM¹ · FP4
Compare curves
B300
$0.134
TRTLLM · FP4
GB200 NVL72
$0.514
Dynamo vLLM · FP4
GB300 NVL72
$0.21317% cheaper than this platform’s Jun 15 result
Dynamo SGLang · FP4STP
Compare curves
DeepSeek V4 Pro 1.6T
Long Context Multi-Turn Realistic Agentic Scenario (AgentX)
B200
$0.050
vLLM · FP4
MI355X
$0.256
MoRI SGLang · FP4
B300
$0.034
vLLM · FP4
GB200 NVL72
—
no data for this scenario
GB300 NVL72
$0.011
Dynamo vLLM · FP4
Kimi K3 2.8T
Long Context Multi-Turn Realistic Agentic Scenario (AgentX)
B200
$1.350
Dynamo vLLM · FP4STP
MI355X
—
no data for this scenario
B300
$0.154
vLLM · FP4
GB200 NVL72
$0.497
Dynamo vLLM · FP4
GB300 NVL72
—
no data for this scenario
MiniMax M3 428B
8K/1K
B200
$0.06713% cheaper than this platform’s Jun 23 result
vLLM · FP8
Compare curves
MI355X
$0.053
ATOM¹ · FP4
B300
$0.05612% cheaper than this platform’s Jul 6 result
vLLM · FP4
Compare curves
GB200 NVL72
$0.10814% more expensive than this platform’s Jun 29 result
Dynamo vLLM · FP8STP
Compare curves
GB300 NVL72
$0.110About the same cost as this platform’s Jun 26 result
Dynamo vLLM · FP8STP
Compare curves
MiniMax M3 428B
Long Context Multi-Turn Realistic Agentic Scenario (AgentX)
B200
—
no exact @50 result
MI355X
—
no data for this scenario
B300
$0.012
vLLM · FP4
GB200 NVL72
—
no data for this scenario
GB300 NVL72
—
no data for this scenario
GLM5.2
Long Context Multi-Turn Realistic Agentic Scenario (AgentX)
B200
—
no exact @50 result
MI355X
—
no data for this scenario
B300
—
no exact @50 result
GB200 NVL72
—
no data for this scenario
GB300 NVL72
—
no data for this scenario
Qwen3.5 397B
8K/1K
B200
$0.036
TRTLLM · FP4
MI355X
$0.06727% cheaper than this platform’s Jun 15 result
SGLang · FP4
Compare curves
B300
$0.0406% cheaper than this platform’s Jul 4 result
SGLang · FP4
Compare curves
GB200 NVL72
$0.142
Dynamo SGLang · FP8STP
GB300 NVL72
$0.06333% cheaper than this platform’s Jun 25 result
Dynamo TRTLLM · FP4STP
Compare curves
Qwen3.5 397B
Long Context Multi-Turn Realistic Agentic Scenario (AgentX)
B200
—
no exact @50 result
MI355X
—
no data for this scenario
B300
$0.006
SGLang · FP4
GB200 NVL72
—
no data for this scenario
GB300 NVL72
$0.007
Dynamo SGLang · FP4
DeepSeek R1 0528 671BMaintenance
8K/1K
B200
$0.077
TRTLLM · FP4
MI355X
$0.157
ATOM¹ · FP4
B300
$0.139
Dynamo TRTLLM · FP4
GB200 NVL72
$0.109
Dynamo TRTLLM · FP4STP
GB300 NVL72
$0.230
Dynamo TRTLLM · FP8STP
Kimi K2.5/2.6/2.7-Code 1TDeprecated
8K/1K
B200
$0.084
Dynamo TRTLLM · FP4STP
MI355X
$0.132
ATOM¹ · FP4STP
B300
$0.179
vLLM · FP4STP
GB200 NVL72
$0.079
Dynamo TRTLLM · FP4STP
GB300 NVL72
$0.098
Dynamo TRTLLM · FP4STP
GLM5/5.1 744BDeprecated
8K/1K
B200
$0.172
SGLang · FP4
MI355X
$0.431
SGLang · FP8
B300
$0.210
SGLang · FP4
GB200 NVL72
$0.26022% cheaper than this platform’s Jul 8 result
Dynamo TRTLLM · FP4STP
Compare curves
GB300 NVL72
$0.281
Dynamo TRTLLM · FP4STP
gpt-oss 120BDeprecated
8K/1K
B200
$0.010
TRTLLM · FP4STP
MI355X
$0.010
ATOM¹ · FP4STP
B300
—
no data for this scenario
GB200 NVL72
—
no exact @50 result
GB300 NVL72
—
no data for this scenario
MiniMax M2.5/2.7 230BDeprecated
8K/1K
B200
$0.041
vLLM · FP4STP
MI355X
$0.051
ATOM¹ · FP4STP
B300
$0.051
vLLM · FP4STP
GB200 NVL72
$0.041
Dynamo vLLM · FP4STP
GB300 NVL72
$0.047
Dynamo vLLM · FP4STP
Llama 3.3 70B InstructDeprecated
8K/1K
B200
$0.065
TRTLLM · FP4STP
MI355X
$0.145
vLLM · FP4STP
B300
—
no data for this scenario
GB200 NVL72
—
no data for this scenario
GB300 NVL72
—
no data for this scenario

Current cost and change versus the latest validated platform result 30–60 days earlier.

Platforms without a valid 30-day comparison show current cost only.

If a chip does not have FP4 spec decoding available, the next best available configuration is used.

DeepSeek V4 Pro 1.6T

DeepSeek V4 Pro 1.6T

Kimi K3 2.8T

MiniMax M3 428B

MiniMax M3 428B

GLM5.2

Qwen3.5 397B

Qwen3.5 397B

DeepSeek R1 0528 671BMaintenance

Kimi K2.5/2.6/2.7-Code 1TDeprecated

GLM5/5.1 744BDeprecated

gpt-oss 120BDeprecated

MiniMax M2.5/2.7 230BDeprecated

Llama 3.3 70B InstructDeprecated

DeepSeek V4 Pro 1.6T

DeepSeek V4 Pro 1.6T

Kimi K3 2.8T

MiniMax M3 428B

MiniMax M3 428B

GLM5.2

Qwen3.5 397B

Qwen3.5 397B

DeepSeek R1 0528 671BMaintenance

Kimi K2.5/2.6/2.7-Code 1TDeprecated

GLM5/5.1 744BDeprecated

gpt-oss 120BDeprecated

MiniMax M2.5/2.7 230BDeprecated

Llama 3.3 70B InstructDeprecated