Top-k gating
Top-k gating selects the k highest-scoring experts for each input and assigns routing weights to the selected experts.
Rearchitecting LLMs
Top-k routing selects the k experts with the highest router scores for each token and normalizes their selected weights.
- How it is used
- Routes each token to selected experts using router scores and combines their outputs with normalized weights.
- How deep it goes
- Implements training and inference routing, including expert selection and output weighting.
- Examples
- Uses a four-expert top-2 routing diagram and Mixtral 8x7B’s eight-expert, top-2 configuration.
- What you can do after
- Build it from scratch by selecting top-k router logits, normalizing their weights, and combining selected expert outputs in a PyTorch forward pass.
Build a DeepSeek Model (From Scratch)
Top-k gating selects the highest-scoring experts for each token and assigns normalized routing weights to them.
- How it is used
- Routes tokens sparsely through selected experts while preserving shared and residual paths in a DeepSeekMoE layer.
- How deep it goes
- Implements logits, biasing, top-k selection, dispatch, weighted recombination, and gradient-free load balancing.
- Examples
- Uses PyTorch listings for router computation, torch.topk selection, masked dispatch, indexed accumulation, and bias updates.
- What you can do after
- Build it from scratch as a sparse PyTorch routing path with normalized gates and dynamic expert-load correction.
Read how Build a DeepSeek Model (From Scratch) explains top-k gating →
What to know first
- model weightA model weight is a learned numerical parameter that influences a machine-learning model’s computations and behavior.
Related concepts
- torch.topkuses · torch.topk is a PyTorch operation that returns the largest or smallest k elements of a tensor together with their indices.
- gatingkind of · Gating is the selective regulation of information flow within a computational system, controlling what information is passed, retained, combined, or suppressed.
- softmaxuses · Softmax is a function that exponentiates each value in a score vector and normalizes the results to produce positive values summing to one, forming a probability distribution.
- routerimplements · A router is a component that directs an incoming request or task to a destination or processing path according to routing criteria.
- expert routingproduces · Expert routing is the mechanism that selects which expert subnetworks process an input or token in a mixture-of-experts model.
- top-kcontrasts with · Top-k is a selection operation that retains the k items ranked highest according to a specified value or criterion.
Questions about top-k gating
What is top-k gating?
Top-k gating selects the k highest-scoring experts for each input and assigns routing weights to the selected experts.
Which Manning books explain top-k gating?
Rearchitecting LLMs and Build a DeepSeek Model (From Scratch).
What should I know before learning top-k gating?
Manning books that teach top-k gating usually introduce model weight first.
What is the difference between top-k gating and top-k?
Top-k gating selects the k highest-scoring experts for each input and assigns routing weights to the selected experts. Top-k is a selection operation that retains the k items ranked highest according to a specified value or criterion.
What is top-k gating a kind of?
Top-k gating is a kind of gating. Gating is the selective regulation of information flow within a computational system, controlling what information is passed, retained, combined, or suppressed.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.

