A routing matrix is a trainable linear layer that maps token representations to one raw routing score per expert. For an input matrix of shape (4, 8) and three experts, the routing matrix has shape (8, 3), producing an expert selector matrix of shape (4, 3). section 4.2.3
Figure 4.7 The routing mechanism. The input matrix is multiplied by a learned routing matrix to produce an expert selector matrix, which contains a raw score for each expert for each token.

The routing mechanism shows the matrix multiplication from token representations to per-token, per-expert scores. section 4.2.3
Interpret the resulting expert selector matrix as follows:
- Each row represents one token, and each column represents one expert. section 4.2.3
- Each entry is a raw, unnormalized suitability score; top-k selection keeps the highest-scoring experts for each token. section 4.2.3 section 4.2.4
- After masking non-selected scores with negative infinity and applying softmax, the selected scores become weights that sum to one. section 4.2.4
Routing one token
For the token "The," suppose Expert 2 receives weight 0.6 and Expert 3 receives weight 0.4. The model combines those experts' outputs as 0.6 × Output_E2 + 0.4 × Output_E3, while ignoring Expert 1. The same weighted combination runs across all tokens, preferably through vectorized scatter/gather operations rather than token-by-token loops. section 4.2.5
The routing matrix is also adjusted to influence expert utilization. An auxiliary loss can penalize variation in expert importance, measured from selector-weight columns with the coefficient of variation. An alternative maintains per-expert biases, increasing underloaded experts' scores and decreasing overloaded experts' scores before top-k selection. section 4.3.1 section 4.4.4
