An auxiliary-loss-free bias update balances mixture-of-experts routing by adjusting expert biases from observed token loads, without adding a balancing loss to gradient training. Called once per optimizer step, it forms biased router logits, selects top-k experts, counts assignments, and applies a bounded correction that raises under-loaded experts and lowers over-loaded ones. section 4.5
The update follows a compact feedback procedure:
- Add bias_lr multiplied by tanh(load violation), producing a smooth bounded correction for the next forward pass. section 4.5
