liveBook · Manning

Logit

As explained in Rearchitecting LLMs

A logit is an unnormalized numerical score produced by a model before conversion into probabilities.

A logit is an unnormalized model score. Applying softmax converts logits into probabilities, which can then be compared during knowledge distillation. section 2.5

In logits-only distillation, the training cycle is:

  • Pass a text batch to the teacher and collect its logits. section 2.5
  • Pass the same batch to the student and collect its logits. section 2.5
  • Convert both outputs to probability distributions, measure their difference with KL divergence, and update the student’s weights. section 2.5
Listing 2.3 Step 2: Getting the teacher's answer(logits)
with torch.no_grad():
    teacher_outputs = teacher_model(
        input_ids=input_ids,
        attention_mask=attention_mask
    )
    teacher_logits = teacher_outputs.logits

See how a teacher model produces logits from input IDs and an attention mask. section 2.5

How Rearchitecting LLMs covers it

How deep it goes
Explains teacher and student inference, softmax conversion, KL-divergence loss, and weight updates.
Examples
Uses teacher and pruned-student outputs for the same text batch as the comparison example.
What you can do after
Apply PyTorch softmax and KL-divergence operations to train a student from teacher outputs.
How it connects
Connects logits to softmax, probability distributions, knowledge distillation, and KL divergence.

Also explained in

Compare how 24 books explain logit →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Rearchitecting LLMs. The section links point to the parts of the book it draws on.