Total loss aggregates multiple training errors into one objective. For multi-token prediction, it sums the individual cross-entropy losses across prediction depths, giving the model gradient signals for immediate and longer-term forecasting. section 5.3.3
Aggregation can also preserve distinct learning signals through weighting. Knowledge distillation combines hard-label cross-entropy with temperature-scaled KL divergence: L = αL_hard + (1−α)T²L_soft. The weights control the balance between ground-truth correctness and teacher guidance. section 8.4.4 section 8.6.2
A total loss supports several practical design choices:
- Sum losses when multiple prediction targets should contribute directly to one training signal. section 5.3.3
- Weight terms when different objectives require controlled influence, such as hard labels versus teacher-provided soft targets. section 8.4.4
- Backpropagate the aggregate objective while updating only the intended trainable model; the frozen teacher contributes targets without receiving gradients. section 8.6.2
Numerical distillation loss
With α=0.3, temperature T=3, hard loss 0.275, and temperature-scaled soft loss 0.179, the total loss is 0.3×0.275 + 0.7×0.179 ≈ 0.208. section 8.6.2
Figure 5.14 The total loss for a single input token is the sum of the individual cross-entropy losses for each of the predicted future tokens.

Visualizes total loss as the sum of individual cross-entropy losses for future-token prediction depths. section 5.3.3
