liveBook · Manning

Expert routing

Expert routing is the mechanism that selects which expert subnetworks process an input or token in a mixture-of-experts model.

How 3 Manning books explain it

CUDA for LLMs

Expert routing selects experts to process inputs in a mixture-of-experts model.

How it is used
Enables sparse expert activation during transformer inference by selecting a few experts per token.
How deep it goes
Traces routing from score normalization through weighted output combination and details a single-thread CUDA top-k scan.
Examples
Works through eight expert logits with top-2 selection and a CUDA-backed MoE forward pass.
What you can do after
Builds a simple CUDA top-k selection kernel for expert routing and applies it in an MoE forward pass.

Read how CUDA for LLMs explains expert routing →

Rearchitecting LLMs

Sparse routing processes each token with only a subset of available expert modules rather than with every expert.

How it is used
Reduces inference compute by sending each token only to its top-k experts.
How deep it goes
Implements top-k selection, weight renormalization, token regrouping, and expert execution, contrasting sparse inference with dense training.
Examples
Uses four experts with top_k=2, leaving half idle per token, and boolean masks to group assignments.
What you can do after
Build it from scratch by selecting experts per token, grouping tokens by expert, and combining weighted outputs.

Read how Rearchitecting LLMs explains expert routing →

Build a DeepSeek Model (From Scratch)

Routing is the mechanism that selects which expert subnetworks process a given input in a mixture-of-experts architecture.

How it is used
Uses learned token-to-expert selection to make mixture-of-experts computation sparse and specialized.
How deep it goes
Traces routing from token selection through expert outputs and matrix combination, without detailing router equations or training procedures.
Examples
Uses “What is 1+1?” and “The next day is.” alongside token-routing and expert-output matrix diagrams.
What you can do after
Evaluate how selective expert activation produces one Transformer-compatible output while avoiding unnecessary expert computation.

Read how Build a DeepSeek Model (From Scratch) explains expert routing →

What to know first

Related concepts

Questions about expert routing

What is expert routing?

Expert routing is the mechanism that selects which expert subnetworks process an input or token in a mixture-of-experts model.

Which Manning books explain expert routing?

CUDA for LLMs, Rearchitecting LLMs and Build a DeepSeek Model (From Scratch).

What should I know before learning expert routing?

Manning books that teach expert routing usually introduce feedforward network, linear layer, softmax and token first.

What kinds of expert routing are there?

Kinds of expert routing include expert-choice routing.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.