EAGLE
Also known as EAGLE speculative decoding, EAGLE-3
In plain English
EAGLE is a particular way to draft several likely next tokens for the main model to check, which can make answers stream faster.
Technical definition
EAGLE is a family of speculative-decoding methods that predicts draft continuations from features associated with the target language model and then verifies them with the target model.
Engineering details
Serving frameworks expose EAGLE through settings such as the number of speculative steps, draft tokens, and candidate width. Model checkpoints and draft components must match the engine implementation.
Why it matters
EAGLE can raise accepted tokens per target-model step, but its result is workload dependent. Acceptance behavior, draft overhead, model architecture, and batch size determine whether the extra path improves end-to-end serving.
How to read it in InferenceX
Some InferenceX curves label the feature MTP because the model supplies multi-token heads while the engine uses EAGLE-style speculative plumbing. The recipe flags and checkpoint details identify the exact implementation.
Source material
See the concept in real benchmarks
B200 NVFP4 vs H200 FP8 on GLM-5: Up to 3.65x Better Performance per Dollar with SGLang MTP
Both SKUs run SGLang EAGLE MTP; the Blackwell generation lifts perf/$ by ~1.2x at the peak and the NVIDIA GLM-5-NVFP4 checkpoint on FlashInfer TRT-LLM sparse MLA stacks another ~2.4–3.0x on 8K/1K
DeepSeekV4 1.6T Day 0 to Day 43 Performance Over Time — Huawei, GB300 NVL72, MI355X, B200
Day 0 Inference Performance, InferenceX, 100x performance improvement in 26 Days, Cost per Million Tokens, Huawei 950DT Inference Trace Analysis