liveBook · Manning

Probability distribution

As explained in Timeless Algorithms, The Foundational Ideas

A probability distribution assigns nonnegative probabilities to possible outcomes so that the probabilities over the defined outcome space sum to one.

A probability distribution assigns probabilities to possible outcomes. When those outcomes form the complete outcome space, their probabilities sum to one. section 2.3.1

Distributions represent uncertainty in predictive models: a classifier allocates probability across possible classes, while a language model assigns probabilities to possible next tokens. The allocation is meaningful only relative to the outcomes the model treats as complete. section 13.4.1 section 11.2

∑i=1nP(Ai)=1

For a complete set of outcomes, the probabilities sum to one. section 2.3.1

Predictive examples show how a distribution represents alternatives:

  • An email classifier assigns 0.3 probability to spam and 0.7 to not spam; together, the probabilities total one. section 2.3.1
  • A diagnostic system assigns probabilities to Disease A, Disease B, and neither, with the complete set totaling one. section 2.3.1

In neural-network training, cross-entropy compares the predicted distribution with the true outcome distribution. It evaluates probability assigned across alternatives, penalizing predictions that spread probability over incorrect outcomes. section 5.4.1

How Timeless Algorithms, The Foundational Ideas covers it

How deep it goes
Defines valid distributions through complete outcome sets summing to one and applies this condition, without developing a general probability formalism.
Examples
Uses spam classification and diagnosis to illustrate probabilities across mutually exclusive, exhaustive outcomes.
What you can do after
Evaluate whether a classifier's probabilities cover its complete set of outcomes and sum to one.
How it connects
Links probability distributions to cross-entropy, neural networks, and language-model prediction.

Also explained in

Compare how 33 books explain probability distribution →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Timeless Algorithms, The Foundational Ideas. The section links point to the parts of the book it draws on.