Inference Performance

Inference performance metrics across different models, hardware configurations, and serving parameters.

Model

ISL / OSL

Precision

Y-Axis Metric

GPU Config

Features:Multi-head Latent AttentionAuxiliary-loss-free Load BalancingMulti-Token PredictionSource

Released by DeepSeek on May 28, 2025