A logits matrix stores raw, unnormalized scores before routing or token selection. In a mixture-of-experts router, multiplying token embeddings by a learned routing matrix produces one score for every token–expert pair. section 4.2.3
Token-to-expert scores
With four token embeddings of shape (4, 8) and three experts, a routing matrix of shape (8, 3) produces a logits matrix of shape (4, 3). Each row represents a token, each column an expert, and each cell its raw suitability score. Top-k routing keeps the highest two scores per row, masks the rest with negative infinity, and applies row-wise softmax to obtain weights that sum to 1. section 4.2.3 section 4.2.4
The same structure also appears during autoregressive language-model inference:
- Select the final row because it contains predictions for the next token after the complete input; earlier rows are unnecessary for that decision. section 2.2.3
- Apply softmax to convert the final logits vector into vocabulary probabilities, then use argmax or controlled sampling to choose a token. section 2.2.3
Figure 4.7 The routing mechanism. The input matrix is multiplied by a learned routing matrix to produce an expert selector matrix, which contains a raw score for each expert for each token.

Visualize the matrix multiplication that turns token representations into expert-specific routing scores. section 4.2.3
