Probability measures how likely an event is. In the classical frequentist view, it is the fraction of all possible outcomes that satisfy the event: favorable outcomes divided by total outcomes. Because an event is contained within the full outcome population, its probability lies from 0 to 1. Probability 0 denotes an impossible event, while probability 1 denotes a certain event; values near either boundary can still describe uncertain events. section 5.1 section 5.3.1
For a city of 100,000 adults where 25,000 are at least 6 ft tall, the probability of selecting such an adult is 25,000/100,000 = 0.25. The same reasoning can describe one condition, such as height, or combined conditions, such as a weight range and a height range. Probability therefore turns population counts into comparable measures of chance. section 5.1
Basic rules organize probabilities across events:
- Mutually exclusive events cannot occur together; the probability that one or another occurs is the sum of their probabilities. section 5.10
- Exhaustive mutually exclusive events cover every possible outcome, so their probabilities sum to 1. section 5.10
- For independent events, the probability that all occur together is the product of their individual probabilities. section 5.10
- A discrete random variable assigns probabilities to particular values; a continuous random variable uses probability density over infinitesimal ranges, whose integral over its domain is 1. section 5.10
Three photo selections
Suppose 20% of photos contain a celebrity, so success has probability π = 0.2 and failure has probability 0.8. Exactly two celebrity photos in three selections can occur as FSS, SFS, or SSF. Each arrangement has probability 0.8 × 0.2 × 0.2, so adding the three mutually exclusive arrangements gives 0.096. This combines multiplication for independent selections with addition across alternative arrangements. section 5.9.3 section 5.3
Table 5.8 All possible combinations of three trials
| Event Id |
Event |
Probability |
|---|---|---|
| 0 |
{F, F, F} |
(1−π) × (1−π) × (1−π) = 0.8 × 0.8 × 0.8 |
| 1 |
{F, F, S} |
(1−π) × (1−π) × π = 0.8 × 0.8 × 0.2 |
| 2 |
{F, S, F} |
(1−π) × π × (1−π) = 0.8 × 0.2 × 0.8 |
| 3 |
{F, S, S} |
(1−π) × π × π = 0.8 × 0.2 × 0.2 |
| 4 |
{S, F, F} |
π × (1−π) × (1−π) = 0.2 × 0.8 × 0.8 |
| 5 |
{S, F, S} |
π × (1−π) × π = 0.2 × 0.8 × 0.2 |
| 6 |
{S, S, F} |
π × π × (1−π) = 0.2 × 0.2 × 0.8 |
| 7 |
{S, S, S} |
π × π × π = 0.2 × 0.2 × 0.2 |
Compare all eight success/failure sequences for three trials to see why the three sequences with exactly two successes are added. section 5.9.3
Probability distributions extend the idea from one event to possible values of a random variable. Sampling returns arbitrary values, but many samples make higher-probability regions denser than lower-probability regions. Their expected value is the large-sample average, equivalently a probability-weighted sum over possible values. section 5.10
In machine learning, model outputs can be probabilities rather than only labels. A classifier can emit probabilities for cat, dog, airplane, and automobile; cross-entropy compares that predicted distribution with a one-hot ground-truth distribution and supplies a training loss. Probability models can also estimate the chance that a face appears in an image or an event falls within a range after distribution parameters are estimated from data. section 6.3 section 6.8.1
