Suppose you think rain is unlikely, then spot a dark cloud. You may now give rain a better chance than you did before. A posterior probability puts a number on that updated belief: it is the probability of a hypothesis after accounting for observed data. The collection of probabilities across hypotheses is the posterior distribution. section 1.5
Bayes’ theorem turns that intuition into a calculation. For each hypothesis, multiply its prior probability—your belief before seeing the data—by the likelihood, or how probable the data would be if that hypothesis were true. These products are not yet probabilities that sum to 1. Divide each by the sum of all the products to get the posterior probabilities: Pr(H | D) = Pr(D | H) × Pr(H) / Pr(D). section 2.2
Three competing hypotheses
Start with prior probabilities 0.4, 0.2, and 0.4. If their likelihoods for the observed data are 0.3, 0.2, and 0.1, multiplication gives 0.12, 0.04, and 0.04. Divide each by their sum, 0.20: the posterior probabilities are 0.6, 0.2, and 0.2. The first hypothesis gains ground because it both started relatively strong and explains the data best. section 2.3
A positive test is not a diagnosis
For a disease affecting 0.7% of people, suppose a test detects 98% of cases but has a 5% false-positive rate. After a positive result, the posterior probability of having the disease is still only about 12%. The disease’s rarity matters alongside the test result; ignoring it is the base-rate fallacy. Further tests can update the probabilities again. section 2.5
A posterior distribution also answers questions beyond “Which value is most likely?” In a five-person survey where three prefer tea, a model’s most likely proportion of tea drinkers is 0.6, but its posterior probability is only about 0.21. Summing posterior probabilities for proportions of at least 0.8 gives about 0.167. Both results matter: a leading estimate need not be a confident one. section 2.12
unnormalized_posterior = prior * likelihood
posterior = (
unnormalized_posterior
/ unnormalized_posterior.sum()
)
The NumPy calculation follows the same two steps as the hand example: multiply corresponding prior and likelihood values, then divide by the sum. It handles eleven candidate proportions at once. section 2.11
Posterior probabilities depend on both prior beliefs and data. With more survey responses showing the same proportion, the posterior concentrates more strongly near 0.6. Different priors can give different answers when data is scarce; a hypothesis assigned zero prior probability cannot gain posterior probability from later data. For models whose posterior cannot be calculated directly, sampling methods can approximate it. section 2.13 section 2.14 section 5.0










