A mixture-of-experts (MoE) model keeps many specialized subnetworks, called experts, but uses only a few for each token. A router selects which experts activate; the rest stay idle. In a dense model, every parameter—the learned numerical values stored in the model—activates for every token. MoE reduces the computation needed to generate a token without making every expert work on it. section 12.5.1 section 12.2.2
The router and experts are not assigned fixed jobs in advance. They start with random weights and learn together during training: the router learns which experts to select, while the experts develop specializations. This creates two useful size measures: total parameters describe the model’s overall capacity, while active parameters indicate how much of it computes each token. section 12.5.1
An 80B model with 3B active
Qwen3-Coder-Next has about 80 billion parameters in total, but activates about 3 billion per token. That smaller active count helps explain its generation speed; it does not mean the whole model needs only 3B-sized storage. One Ollama build is approximately 52 GB to download, and running it needs additional memory for context and overhead. Check the build and available RAM before downloading. section 12.5.3 section 17.3.1
Table 12.5 Key specifications of Qwen3-Coder-Next
| Specification |
Value |
|---|---|
| Total parameters |
80 billion |
| Active parameters |
3 billion |
| Expert count |
512 (10 active per token + 1 shared) |
| Context window |
256,000 tokens |
| License |
Open |
| Architecture |
Ultra-sparse MoE |
Compare Qwen3-Coder-Next’s total and active parameters, then see how many experts its ultra-sparse design uses per token. section 12.5.3
