35 min read
TPU Inference Externalization Full Steam Ahead
InferenceX, Up to 50% Better Performance per Dollar, Rapid Externalization of TPU stack, Growing Customer Base, Ironwood, TPUv8i, Reducing CUDA Moat
- benchmark
- inference
- tpu
- +7
InferenceX Research
Articles
Benchmark write-ups on agentic inference, AgentX results, and chip and serving-stack economics.
New to the terminology? Browse the AI inference glossary.
Tagged “b200”
16 articles
3 min read
DeepSeek V4 Pro on AgentX: B200 vs B300 and the KV Cache Working Set
50% more HBM squeezes out extra throughput, and the per-point telemetry shows exactly where it comes from
- agentx
- agentic
- benchmark
- +6
3 min read
DeepSeek V4 Pro on AgentX: MI355X vs B200, and the August 21 Flip
AMD matched B200 vLLM on performance per dollar for end-to-end latency, then upstream vLLM work moved the line
- agentx
- agentic
- benchmark
- +6
2 min read
GLM 5.3 on AgentX: NVIDIA Is Up to 5x Cheaper per Token at 150 tok/s/user
At this operating point, free AMD silicon would still not close the gap
- agentx
- agentic
- benchmark
- +8
3 min read
MiniMax M3 on AgentX: Why B200 and B300 Beat Their Rack-Scale GB200 NVL72 & GB300 NVL72 Counterparts
The Dynamo router becomes the bottleneck, no submission runs context parallelism, and AMD leaves KV offload on the table
- agentx
- agentic
- benchmark
- +10
2 min read
MiniMax M3 on AgentX: B300 TRT-LLM TP2 Owns the Crown
NVIDIA sweeps the 432B model, and the missing DP-attention points explain why cache locality became a routing constraint
- agentx
- agentic
- benchmark
- +8
17 min read
Ultra-High Interactivity on NVIDIA GPUs? TileRT on InferenceX
Can TileRT software on NVIDIA GPUs compete with Cerebras, Groq LPU, and SambaNova? Batch size 1, disaggregated engine, high-throughput prefill engine, high-interactivity decode engine
- benchmark
- gpu
- inference
- +8
23 min read
Kimi K3: The Manos, The Mythos, The Legendos
Kimi K3's architecture: compressed memory, attention across depth, latent expert routing, and serving performance
- inference
- benchmark
- gpu
- +6
29 min read
DeepSeekV4 1.6T Day 0 to Day 43 Performance Over Time — Huawei, GB300 NVL72, MI355X, B200
Day 0 Inference Performance, InferenceX, 100x performance improvement in 26 Days, Cost per Million Tokens, Huawei 950DT Inference Trace Analysis
- benchmark
- gpu
- inference
- +13
10 min read
B200 NVFP4 vs H200 FP8 on GLM-5: Up to 3.65x Better Performance per Dollar with SGLang MTP
Both SKUs run SGLang EAGLE MTP; the Blackwell generation lifts perf/$ by ~1.2x at the peak and the NVIDIA GLM-5-NVFP4 checkpoint on FlashInfer TRT-LLM sparse MLA stacks another ~2.4–3.0x on 8K/1K
- benchmark
- gpu
- inference
- +6
12 min read
B200 NVFP4 vs H100 FP8 on MiniMax-M2.5: Up to 8.2x Better Performance per Dollar with vLLM
vLLM PR #36307 unlocks the trtllm-gen FP8 MoE kernel for MiniMax on B200; combined with NVFP4, perf/$ scales from 4.0x at 22 tok/s/user to 8.2x at 110 on 8K/1K
- benchmark
- gpu
- inference
- +6
11 min read
B200 NVFP4 vs H200 INT4 on Kimi K2.5/K2.6: Up to 2.95x Better Performance per Dollar
On vLLM 8K/1K the NVFP4 path on B200 is 2.71x–2.95x cheaper per million tokens than H200 INT4 across the entire 30–90 tok/s/user serving band, and 2.45x–2.74x cheaper than B200 INT4 on the same silicon. Both factors decompose cleanly into B200's HBM bandwidth, HBM capacity, and NVFP4 tensor cores
- benchmark
- gpu
- inference
- +6
7 min read
AMD MI355X GLM-5 Inference: Up to 40% Cheaper per Million Tokens than B200 on SGLang FP8
14 weeks after GLM-5 launched, AMD landed both MTP and non-MTP SGLang FP8 recipes on MI355X — fused MLA + FP8 KV cache via TileLang flips the single-node FP8 cost curve in AMD favor across most of the performance Pareto
- benchmark
- gpu
- inference
- +7
8 min read
GB200 NVL72 vs B200 on DeepSeek R1 670B: Up to 4.4x Throughput per GPU at 125 tok/s/user
DeepSeek R1 FP4 1k/1k. NVL72's 72-GPU NVLink scale-up fabric lets decode run wide EP up to EP=32, where B200's 8-GPU NVLink island caps out at EP=8 over RoCEv2
- benchmark
- gpu
- inference
- +9
5 min read
SGLang 0.5.6 on B200 DeepSeek R1 FP4: Up to 1.8x at Low Concurrency
Piecewise CUDA graphs for DeepSeek V3, a unified event loop, and JIT kernels push 8k/1k throughput from 508 to 907 tok/s/GPU on the same 16 GPU B200 pool
- benchmark
- inference
- gpu
- +5
6 min read
GB200 NVL72 vs B200 on Kimi K2.5: 3.1x from Wide EP vLLM
Rack scale NVLink on NVL72 lets Dynamo vLLM run Kimi K2.5 wide EP up to Decode EP 16, taking peak throughput from 4,021 to 12,587 tok/s/GPU on 8k/1k NVFP4
- benchmark
- gpu
- inference
- +7