liveBook · Manning

Routing

As explained in Machines that Think

Routing is the process of assigning each input token to selected experts in a mixture-of-experts architecture.

Routing assigns each input token to a small subset of experts in a mixture-of-experts (MoE) layer. Unlike uniform processing, different tokens can follow different internal paths. section 4.4.2 section 4.4.3

For each token, a gating network scores the experts using the token’s current representation. It selects the top few and converts their scores into weights for their contributions to the layer output. Selection also determines which experts receive a direct learning signal for that token. Repeated assignments reinforce useful specializations, without requiring experts to be given explicit roles. section 4.4.2 section 4.4.2

Routing has to balance specialization against training coverage and system efficiency:

  • Concentrating assignments too heavily can leave some experts with too little data; distributing them too evenly can weaken specialization. section 4.4.2
  • Different routes create uneven workloads across devices, complicating batching, scheduling, and hardware utilization. section 4.4.3

MoE layers are generally used selectively, often in place of Transformer feedforward blocks, while attention layers remain unchanged. Routing can therefore support greater model capacity without increasing computation per token in proportion to the number of experts. section 4.4.3 section 4.4.3

A learned division of labor

A family doctor who improves at referring patients based on feedback illustrates the loop: routing patterns help specialists develop, and their improved performance makes similar referrals more useful. section 4.4.2

How Machines that Think covers it

How deep it goes
Explains token scoring, top-expert selection, weighting, learning signals, and balancing constraints, but gives no routing algorithm or equations.
Examples
Uses a family doctor referring patients to specialists as an analogy for learned routing and specialization.
What it assumes
Assumes an MoE layer is in place and describes how routing allocates tokens among its experts.
What you can do after
Understand how token assignment shapes expert learning and affects computational workloads.
How it connects
Connects routing to gating networks, expert specialization, mixture-of-experts layers, and Transformer feedforward blocks.

Also explained in

Compare how 34 books explain routing →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Machines that Think. The section links point to the parts of the book it draws on.