liveBook · Manning

Logit loss

As explained in Rearchitecting LLMs

Logit loss is a distillation loss that measures disagreement between student and teacher logits or their derived output distributions.

Logit loss helps a student recover behavior from a teacher during distillation. Rather than learning only the dataset’s correct next token, the student also learns to match the teacher’s probability distribution across the vocabulary. section 6.3.1 section 6.6

For next-token distillation, calculate the loss in stages:

  • Divide both models’ logits by the same temperature. Apply log-softmax to the student logits and softmax to the teacher logits. section 6.3.1
  • Drop the final logit position from both outputs, then flatten the remaining positions while retaining the vocabulary dimension. section 6.3.1
  • Calculate KL divergence using the student log-probabilities and teacher probabilities, with batchmean reduction. Multiply the result by temperature squared. section 6.3.1
student_soft = F.log_softmax(
    student_logits / temperature, 
    dim=-1) #A
teacher_soft = F.softmax(
    teacher_logits / temperature, 
    dim=-1) #B
 
student_soft = student_soft[..., :-1, :].contiguous()
teacher_soft = teacher_soft[..., :-1, :].contiguous()
 
loss_logits = F.kl_div(
    student_soft.view(-1, student_soft.size(-1)),
    teacher_soft.view(-1, teacher_soft.size(-1)),
    reduction='batchmean' #C
) * (temperature ** 2) #D

The listing puts temperature scaling, position slicing, reshaping, and KL divergence together. section 6.3.1

Logit loss can be combined with task loss and hidden loss. One baseline weights them 0.4, 0.4, and 0.2 respectively. Adjusting the weights changes the emphasis: generative models prioritize output behavior, while representation-focused models may need more hidden-state alignment. section 6.3.1 section 6.8

Hard labels are not always necessary. Reported large-scale comparisons found that distillation retraining outperformed standard fine-tuning; Mistral also reported that a pure logits objective outperformed its tested combinations with next-token prediction loss. For a specific task, retaining hard-label loss can instead emphasize task results while logits help stabilize learning and generalization. section 6.6

How Rearchitecting LLMs covers it

How deep it goes
Implements temperature-scaled KL divergence and combines it with weighted task and hidden losses.
Examples
Uses a next-token prediction loss implementation and reports Minitron and Mistral distillation comparisons.
What you can do after
Apply it via PyTorch to combine teacher-distribution matching with task and hidden losses.
How it connects
Connects knowledge distillation with soft labels, cross-entropy, KL divergence, and hidden-state alignment.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Rearchitecting LLMs. The section links point to the parts of the book it draws on.