Softmax converts scores across mutually exclusive, collectively exhaustive outcomes into probabilities that sum to one. This lets models express confidence and rank or sample possible outcomes. In attention, it turns dot products into weights that help construct contextual representations. section 2.4.2 section 12.2
From scores to predictions
A language model applies softmax to scores for vocabulary words to produce probabilities for the next word. In a transformer attention calculation, softmax turns dot products into weights; a worked example uses these steps for pronoun resolution. section 2.4.2 section 12.2

























