Chip speculative decoding comparisons
69 speculative decoding comparisons across DeepSeek V4 Pro 0813 1.6T, DeepSeek R1, Kimi K3.1 Pro 2.8T, Kimi K3 2.8T, GLM 5/5.1, MiniMax M3 428B, and Qwen 3.5 397B-A17B. Each page compares inference with the speculative decoding method (MTP, EAGLE and others) enabled versus disabled on the same model and chip, covering throughput, cost and interactivity at matched operating points.
DeepSeek V4 Pro 0813 1.6T
8 speculative decoding comparisons with benchmark data on DeepSeek V4 Pro 0813 1.6T
B200 FP4 — MTP vs Off
NVIDIA · Blackwell
B300 FP4 — MTP vs Off
NVIDIA · Blackwell
GB200 NVL72 FP4 — MTP vs Off
NVIDIA · Blackwell
GB300 NVL72 FP4 — MTP vs Off
NVIDIA · Blackwell
H200 FP8 — MTP vs Off
NVIDIA · Hopper
MI300X FP8 — MTP vs Off
AMD · CDNA 3
MI325X FP8 — MTP vs Off
AMD · CDNA 3
MI355X FP4 — MTP vs Off
AMD · CDNA 4
DeepSeek R1
13 speculative decoding comparisons with benchmark data on DeepSeek R1
B200 FP4 — MTP vs Off
NVIDIA · Blackwell
B200 FP8 — MTP vs Off
NVIDIA · Blackwell
B300 FP4 — MTP vs Off
NVIDIA · Blackwell
B300 FP8 — MTP vs Off
NVIDIA · Blackwell
GB200 NVL72 FP4 — MTP vs Off
NVIDIA · Blackwell
GB200 NVL72 FP8 — MTP vs Off
NVIDIA · Blackwell
GB300 NVL72 FP4 — MTP vs Off
NVIDIA · Blackwell
GB300 NVL72 FP8 — MTP vs Off
NVIDIA · Blackwell
H100 FP8 — MTP vs Off
NVIDIA · Hopper
H200 FP8 — MTP vs Off
NVIDIA · Hopper
MI325X FP8 — MTP vs Off
AMD · CDNA 3
MI355X FP4 — MTP vs Off
AMD · CDNA 4
MI355X FP8 — MTP vs Off
AMD · CDNA 4
Kimi K3.1 Pro 2.8T
1 speculative decoding comparison with benchmark data on Kimi K3.1 Pro 2.8T
Kimi K3 2.8T
6 speculative decoding comparisons with benchmark data on Kimi K3 2.8T
GLM 5/5.1
11 speculative decoding comparisons with benchmark data on GLM 5/5.1
B200 FP4 — MTP vs Off
NVIDIA · Blackwell
B200 FP8 — MTP vs Off
NVIDIA · Blackwell
B300 FP4 — MTP vs Off
NVIDIA · Blackwell
B300 FP8 — MTP vs Off
NVIDIA · Blackwell
GB200 NVL72 FP4 — MTP vs Off
NVIDIA · Blackwell
GB200 NVL72 FP8 — MTP vs Off
NVIDIA · Blackwell
GB300 NVL72 FP4 — MTP vs Off
NVIDIA · Blackwell
GB300 NVL72 FP8 — MTP vs Off
NVIDIA · Blackwell
H200 FP8 — MTP vs Off
NVIDIA · Hopper
MI325X FP8 — MTP vs Off
AMD · CDNA 3
MI355X FP8 — MTP vs Off
AMD · CDNA 4
MiniMax M3 428B
11 speculative decoding comparisons with benchmark data on MiniMax M3 428B
B200 FP4 — M3 EAGLE vs Off
NVIDIA · Blackwell
B200 FP8 — M3 EAGLE vs Off
NVIDIA · Blackwell
B300 FP4 — M3 EAGLE vs Off
NVIDIA · Blackwell
B300 FP8 — M3 EAGLE vs Off
NVIDIA · Blackwell
GB300 NVL72 FP8 — M3 EAGLE vs Off
NVIDIA · Blackwell
H100 FP8 — M3 EAGLE vs Off
NVIDIA · Hopper
H200 FP8 — M3 EAGLE vs Off
NVIDIA · Hopper
MI300X FP8 — M3 EAGLE vs Off
AMD · CDNA 3
MI325X FP8 — M3 EAGLE vs Off
AMD · CDNA 3
MI355X FP4 — M3 EAGLE vs Off
AMD · CDNA 4
MI355X FP8 — M3 EAGLE vs Off
AMD · CDNA 4
Qwen 3.5 397B-A17B
19 speculative decoding comparisons with benchmark data on Qwen 3.5 397B-A17B
B200 FP4 — MTP vs Off
NVIDIA · Blackwell
B200 FP8 — MTP vs Off
NVIDIA · Blackwell
B200 BF16 — MTP vs Off
NVIDIA · Blackwell
B300 FP4 — MTP vs Off
NVIDIA · Blackwell
B300 FP8 — MTP vs Off
NVIDIA · Blackwell
B300 BF16 — MTP vs Off
NVIDIA · Blackwell
GB200 NVL72 FP8 — MTP vs Off
NVIDIA · Blackwell
GB300 NVL72 FP4 — MTP vs Off
NVIDIA · Blackwell
GB300 NVL72 FP8 — MTP vs Off
NVIDIA · Blackwell
H100 FP8 — MTP vs Off
NVIDIA · Hopper
H200 FP8 — MTP vs Off
NVIDIA · Hopper
MI300X FP8 — MTP vs Off
AMD · CDNA 3
MI325X FP8 — MTP vs Off
AMD · CDNA 3
MI325X BF16 — MTP vs Off
AMD · CDNA 3
MI355X FP4 — MTP vs Off
AMD · CDNA 4
MI355X FP8 — MTP vs Off
AMD · CDNA 4
MI355X BF16 — MTP vs Off
AMD · CDNA 4
RTX PRO 6000 FP4 — MTP vs Off
NVIDIA · Blackwell
TPU7x FP8 — MTP vs Off
Google · Ironwood