liveBook · Manning

Top-k gating

Top-k gating selects the k highest-scoring experts for each input and assigns routing weights to the selected experts.

How 2 Manning books explain it

Rearchitecting LLMs

Top-k routing selects the k experts with the highest router scores for each token and normalizes their selected weights.

How it is used
Routes each token to selected experts using router scores and combines their outputs with normalized weights.
How deep it goes
Implements training and inference routing, including expert selection and output weighting.
Examples
Uses a four-expert top-2 routing diagram and Mixtral 8x7B’s eight-expert, top-2 configuration.
What you can do after
Build it from scratch by selecting top-k router logits, normalizing their weights, and combining selected expert outputs in a PyTorch forward pass.

Read how Rearchitecting LLMs explains top-k gating →

Build a DeepSeek Model (From Scratch)

Top-k gating selects the highest-scoring experts for each token and assigns normalized routing weights to them.

How it is used
Routes tokens sparsely through selected experts while preserving shared and residual paths in a DeepSeekMoE layer.
How deep it goes
Implements logits, biasing, top-k selection, dispatch, weighted recombination, and gradient-free load balancing.
Examples
Uses PyTorch listings for router computation, torch.topk selection, masked dispatch, indexed accumulation, and bias updates.
What you can do after
Build it from scratch as a sparse PyTorch routing path with normalized gates and dynamic expert-load correction.

Read how Build a DeepSeek Model (From Scratch) explains top-k gating →

What to know first

Related concepts

Questions about top-k gating

What is top-k gating?

Top-k gating selects the k highest-scoring experts for each input and assigns routing weights to the selected experts.

Which Manning books explain top-k gating?

Rearchitecting LLMs and Build a DeepSeek Model (From Scratch).

What should I know before learning top-k gating?

Manning books that teach top-k gating usually introduce model weight first.

What is the difference between top-k gating and top-k?

Top-k gating selects the k highest-scoring experts for each input and assigns routing weights to the selected experts. Top-k is a selection operation that retains the k items ranked highest according to a specified value or criterion.

What is top-k gating a kind of?

Top-k gating is a kind of gating. Gating is the selective regulation of information flow within a computational system, controlling what information is passed, retained, combined, or suppressed.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.