A probability distribution assigns probabilities to possible outcomes. When those outcomes form the complete outcome space, their probabilities sum to one. section 2.3.1
Distributions represent uncertainty in predictive models: a classifier allocates probability across possible classes, while a language model assigns probabilities to possible next tokens. The allocation is meaningful only relative to the outcomes the model treats as complete. section 13.4.1 section 11.2
For a complete set of outcomes, the probabilities sum to one. section 2.3.1
Predictive examples show how a distribution represents alternatives:
- An email classifier assigns 0.3 probability to spam and 0.7 to not spam; together, the probabilities total one. section 2.3.1
- A diagnostic system assigns probabilities to Disease A, Disease B, and neither, with the complete set totaling one. section 2.3.1
In neural-network training, cross-entropy compares the predicted distribution with the true outcome distribution. It evaluates probability assigned across alternatives, penalizing predictions that spread probability over incorrect outcomes. section 5.4.1
































