Entropy measures uncertainty in the next-token distribution during generation. High entropy spreads probability across many tokens and supports exploration; low entropy concentrates probability on one token, producing deterministic behavior and potentially signaling training collapse. section 7.3.2
Interpret entropy values as a training diagnostic:
- Very low entropy, approximately 0–0.5, indicates that one token dominates. section 7.3.2
- Moderate entropy, approximately 1–2, indicates probabilities shared across a reasonably small token set during stable training. section 7.3.2
- High entropy approaches the logarithm of the vocabulary size and indicates broad uncertainty. section 7.3.2
Computing token and sequence entropy
Convert logits with torch.softmax() and torch.log_softmax(), then calculate entropy as the negative sum of probability–log-probability products. For generated answers, average per-step entropy after the prompt and compare it with the sequence log-probability. section 7.3.2
Figure 7.7 Log probability (logprob) computation of a single token ("this") in the LLM’s generated answer. The LLM returns the logits of the token, which are then converted to softmax probability values via torch.softmax() or logprob values via torch.log_softmax().
Trace how logits become probabilities and log-probabilities before forming the entropy metric. section 7.3.2
