liveBook · Manning

Bayes' theorem

As explained in Grokking Bayes

Bayes' theorem relates conditional probabilities, allowing a posterior probability to be calculated from a prior probability and likelihood.

Suppose you think rain is unlikely, then notice dark clouds. How much should that observation change your mind? Bayes’ theorem updates the probability of each possible hypothesis using the data you observed. section 2.1

For a hypothesis H and observed data D, the rule is Pr(H | D) = Pr(D | H) × Pr(H) / Pr(D). The prior, Pr(H), is your starting probability. The likelihood, Pr(D | H), says how probable the data would be if H were true. The posterior, Pr(H | D), is your updated probability. The evidence, Pr(D), makes the posterior probabilities sum to 1. section 2.2

To update a set of competing hypotheses:

  • Assign each hypothesis a prior probability. section 2.2
  • Find the likelihood of the observed data under each hypothesis. section 2.2
  • Multiply each prior by its likelihood, then divide each product by the sum of all the products. section 2.2

Why a positive test need not mean disease

Suppose a disease affects about 1 in 150 people. A test detects it 98% of the time but returns a false positive for 5% of people without it. For a person with no related prior health conditions, a positive result raises the probability of disease to about 12%, not 98%. The disease’s low starting probability still matters; overlooking it is the base-rate fallacy. section 2.5

Updating beverage preferences

To estimate the proportion of people who prefer tea, consider eleven possible proportions from 0 to 1. Start with equal prior probabilities, then use a survey in which three of five people prefer tea to calculate each likelihood. Multiplying and normalizing produces a posterior that peaks near 0.6. NumPy arrays let the same calculation handle all eleven hypotheses at once. section 2.11

unnormalized_posterior = prior * likelihood
posterior = (
    unnormalized_posterior
    / unnormalized_posterior.sum()
)

These two lines turn the tea survey’s prior and likelihood arrays into posterior probabilities: multiply matching entries, then divide by their total. section 2.11

The starting choices matter. A plausible hypothesis given zero prior probability cannot gain posterior probability, even when data favor it; zero likelihood also rules it out. Include the possibilities you care about rather than excluding one prematurely. With a lot of data, many different priors yield similar posteriors. section 2.15 section 2.14

How Grokking Bayes covers it

How deep it goes
Works through discrete-hypothesis calculations and NumPy normalization; defers continuous distributions.
Examples
Uses dark clouds, a rare-disease test, and a tea-versus-coffee survey to make updates concrete.
What it assumes
Introduces hypotheses and the theorem’s terms while referring back to introductory conditional probability.
What you can do after
Build it from scratch for discrete hypotheses, then calculate a survey posterior with NumPy and SciPy.
How it connects
Connects prior probabilities, likelihoods, posterior distributions, conditional probability, and the law of total probability.

Also explained in

Compare how 3 books explain Bayes' theorem →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Grokking Bayes. The section links point to the parts of the book it draws on.