liveBook · Manning

Posterior probability

As explained in Grokking Bayes

A posterior probability is the probability assigned to a hypothesis after incorporating observed data.

Suppose you think rain is unlikely, then spot a dark cloud. You may now give rain a better chance than you did before. A posterior probability puts a number on that updated belief: it is the probability of a hypothesis after accounting for observed data. The collection of probabilities across hypotheses is the posterior distribution. section 1.5

Bayes’ theorem turns that intuition into a calculation. For each hypothesis, multiply its prior probability—your belief before seeing the data—by the likelihood, or how probable the data would be if that hypothesis were true. These products are not yet probabilities that sum to 1. Divide each by the sum of all the products to get the posterior probabilities: Pr(H | D) = Pr(D | H) × Pr(H) / Pr(D). section 2.2

Three competing hypotheses

Start with prior probabilities 0.4, 0.2, and 0.4. If their likelihoods for the observed data are 0.3, 0.2, and 0.1, multiplication gives 0.12, 0.04, and 0.04. Divide each by their sum, 0.20: the posterior probabilities are 0.6, 0.2, and 0.2. The first hypothesis gains ground because it both started relatively strong and explains the data best. section 2.3

A positive test is not a diagnosis

For a disease affecting 0.7% of people, suppose a test detects 98% of cases but has a 5% false-positive rate. After a positive result, the posterior probability of having the disease is still only about 12%. The disease’s rarity matters alongside the test result; ignoring it is the base-rate fallacy. Further tests can update the probabilities again. section 2.5

A posterior distribution also answers questions beyond “Which value is most likely?” In a five-person survey where three prefer tea, a model’s most likely proportion of tea drinkers is 0.6, but its posterior probability is only about 0.21. Summing posterior probabilities for proportions of at least 0.8 gives about 0.167. Both results matter: a leading estimate need not be a confident one. section 2.12

unnormalized_posterior = prior * likelihood
posterior = (
    unnormalized_posterior
    / unnormalized_posterior.sum()
)

The NumPy calculation follows the same two steps as the hand example: multiply corresponding prior and likelihood values, then divide by the sum. It handles eleven candidate proportions at once. section 2.11

Posterior probabilities depend on both prior beliefs and data. With more survey responses showing the same proportion, the posterior concentrates more strongly near 0.6. Different priors can give different answers when data is scarce; a hypothesis assigned zero prior probability cannot gain posterior probability from later data. For models whose posterior cannot be calculated directly, sampling methods can approximate it. section 2.13 section 2.14 section 5.0

How Grokking Bayes covers it

How deep it goes
Works through multiplication and normalization by hand, then applies the calculation to eleven hypotheses with NumPy; sampling addresses harder posteriors.
Examples
Uses competing hypotheses, rain forecasts, a rare-disease test, and a tea-preference survey with posterior sums.
What it assumes
Builds from initial beliefs, observed data, and small hand calculations before moving to programmed updates.
What you can do after
Practice/perform posterior updates and interpret probabilities for individual hypotheses or groups of possible values.
How it connects
Connects Bayes’ theorem, prior probability, likelihood, normalization, posterior distributions, and Monte Carlo inference.

Also explained in

Compare how 11 books explain posterior probability →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Grokking Bayes. The section links point to the parts of the book it draws on.