Active parameters
Also known as activated parameters, active params
In plain English
Active parameters are the weights a mixture of experts model actually uses for each token, a small slice of its much larger total size.
Technical definition
Active parameters are the subset of a sparse model, its shared layers plus the experts its router selects, that participate in computing any single token.
Engineering details
A mixture of experts model might hold a trillion total parameters while routing each token through only a few tens of billions. Compute per token scales with the active count, which is why sparse frontier models can be affordable to run. Memory tells a different story: every expert must sit in HBM ready to be selected, so capacity requirements and parallelism plans follow total parameters even though arithmetic follows active ones.
Why it matters
The total versus active split explains most modern serving economics. It is why trillion parameter models are deployable at all, why expert parallelism across many chips exists, and why comparing models by total parameter count says little about their serving cost.
How to read it in InferenceX
The MoE models InferenceX benchmarks, including the DeepSeek, Kimi, Qwen, and MiniMax families, all have low active to total ratios, and their recipes spread experts across nodes precisely because total parameters set the memory bill.
Source material
See the concept in real benchmarks
DeepSeekV4 1.6T Day 0 to Day 43 Performance Over Time — Huawei, GB300 NVL72, MI355X, B200
Day 0 Inference Performance, InferenceX, 100x performance improvement in 26 Days, Cost per Million Tokens, Huawei 950DT Inference Trace Analysis
Kimi K3: The Manos, The Mythos, The Legendos
Kimi K3's architecture: compressed memory, attention across depth, latent expert routing, and serving performance