Mixture of Experts (MoE) architecture separates a large language model's total parameter count from its per-token activated parameter count, letting models such as Qwen3 235B-A22B run inference at roughly the speed of a much smaller dense model while keeping a large model's knowledge capacity — the trade-off is that memory requirements still scale with total parameters, not activated ones.
What Is a Mixture of Experts (MoE) Architecture?
Mixture of Experts (MoE) is a large language model architecture in which a model is built from many "expert" sub-networks, but only a small subset of those experts is activated on each inference pass rather than the full parameter setCITE:E1.
The defining feature of MoE is the separation between a model's "total parameters" and its "activated parameters": Qwen3 235B-A22B carries 235 billion total parameters but activates only 22 billion per pass, while DeepSeek V4-Pro carries 1.6 trillion total parameters but activates only 49 billion per passCITE:E2. This split allows overall model capacity to grow far beyond what the per-token compute cost alone would suggest.
Why Does MoE Make Large Model Inference Both Cheaper and Faster?
Inference speed in an MoE model tracks its activated-parameter count, not its total-parameter count, so a 235-billion-parameter MoE model can run at roughly the speed of a 22-billion-parameter dense model while retaining a large model's knowledge capacityCITE:E3.
What Advantage Does MoE Offer for Local Deployment?
MoE benefits local deployment because system memory must hold the full set of parameters (weights), while compute bandwidth only needs to serve the activated subset, so devices with large memory pools — such as Apple Silicon Macs with unified memory — can hold large MoE models in memory and still run them quicklyCITE:E4.
What Are MoE's Limitations and Trade-offs?
MoE reduces compute cost per token but does not reduce memory footprint: fitting a model's full total-parameter count still requires memory sized to that total, and trillion-parameter-class models such as Kimi K2 and DeepSeek V4-Pro exceed single-machine memory limits even after 4-bit quantization, meaning they cannot run locallyCITE:E5. This is the architecture's core trade-off — it saves compute, not memory.
Why Do Mainstream Open-Weight Models in 2026 Adopt MoE?
Most mainstream open-weight models released in 2026 — including DeepSeek V4, Qwen3, Kimi K2, and GLM — use MoE architecture, reflecting an industry-wide approach to reconciling large-model capability with controlled inference costCITE:E6.
Total vs. Activated Parameters, By Model
| Model | Total Parameters | Activated Parameters |
|---|
| Qwen3 235B-A22B | 235B | 22BCITE:E2 |
| DeepSeek V4-Pro | 1.6T | 49BCITE:E2 |
| Kimi K2 | Trillion-parameter class | Exceeds single-machine memory even at 4-bit quantizationCITE:E5 |
What this means: The same activated-parameter mechanism that lets Qwen3 235B-A22B and DeepSeek V4-Pro run inference at the speed of much smaller dense modelsCITE:E2CITE:E3 is also what makes them fit on memory-rich local devices like Apple Silicon MacsCITE:E4. But that mechanism only addresses compute — it does nothing for memory, which is why Kimi K2 and DeepSeek V4-Pro, both trillion-parameter-class, still exceed single-machine memory limits after 4-bit quantizationCITE:E5. That compute/memory split is the specific trade-off behind why DeepSeek V4, Qwen3, Kimi K2, and GLM all converged on MoE as open-weight releases in 2026CITE:E6.