A gating weight is a normalized routing coefficient that determines how strongly a selected expert contributes to an input token. In a mixture-of-experts router, softmax converts the original, bias-free router logits into the weights applied to the chosen experts. Load-balancing bias changes which experts win top-k selection, but does not distort the gating weights those experts receive. section 4.4.4
Implementation implications:
- Compute gating weights from the original router scores. section 4.4.4
- Add the adaptive expert bias only when selecting the top-k experts. section 4.4.4
- Keep routing control separate from the contribution weights applied by selected experts. section 4.4.4
Balancing expert selection
If Experts 1 and 2 are underloaded, their biases increase; if Expert 3 is overloaded, its bias decreases. Future top-k selection shifts accordingly, while the selected experts still receive weights computed from the original logits. section 4.4.4
Figure 4.26 The bias term is added to the raw router logits before the top-k selection process.

Trace where the bias enters the router: before top-k selection, not before gating-weight computation. section 4.4.4
