liveBook · Manning

Posterior probability

As explained in Timeless Algorithms, The Foundational Ideas

A posterior is a probability or probability distribution updated after incorporating observed evidence.

A positive medical test raises the probability of disease, but it does not establish a diagnosis. The probability after incorporating the test result is a posterior probability: a revised assessment that combines what was known beforehand with the evidence observed. section 10.2.1 section 2.2.1

Bayes’ theorem expresses this update as P(H | D) = P(D | H) × P(H) / P(D). Here H is a hypothesis and D is observed data. P(H) is the prior probability; P(D | H) is the likelihood of the data under the hypothesis; and P(D) is the probability of the data across competing hypotheses. The denominator normalizes the result. The posterior P(H | D) therefore depends on both the prior and how well the hypothesis explains the evidence. section 2.2.1

A positive test for a rare disease

Suppose a disease affects 1 in 1,000 people, a test detects 99% of cases, and 1% of healthy people test positive. The probability of a positive result is (0.99 × 0.001) + (0.01 × 0.999) = 0.01098. Given a positive result, the posterior probability of disease is (0.99 × 0.001) / 0.01098, or approximately 9%. The result is informative, but the low base rate and false positives leave substantial uncertainty. section 2.2.2 section 10.2.1

A posterior can also be a distribution over an unknown probability rather than a single probability assigned to a hypothesis. With a Beta prior and a Binomial likelihood, the posterior is another Beta distribution. Its parameters are updated by adding observed defects and non-defects to the corresponding prior parameters. section 7.4.2

Updating a defect-rate distribution

A manufacturer represents uncertainty about a machine’s defect rate with a Beta(1, 19) prior, whose mean is 5%. After inspecting 10 independent parts and finding two defects, the posterior is Beta(3, 27), with mean 10%. The distribution retains uncertainty about the rate rather than treating that mean as a definitive value. section 7.4.2

A posterior reflects the prior, the likelihood model, and the evidence included in the update. As further evidence arrives, beliefs can be revised again; a current posterior need not settle the question. section 2.2.1 section 2.2 section 10.2.1

How Timeless Algorithms, The Foundational Ideas covers it

How deep it goes
Explains Bayes’ theorem and its normalization, then works through a diagnostic calculation and a conjugate posterior distribution.
Examples
Calculates disease probability after a positive test and updates a machine’s defect-rate distribution after inspection.
What you can do after
Evaluate and understand how prior information and observed evidence shape uncertainty in diagnostic and defect-rate estimates.
How it connects
Connects posterior probability to prior probability, likelihood, marginal probability, Bayesian inference, and conjugate distributions.

Also explained in

Compare how 11 books explain posterior probability →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Timeless Algorithms, The Foundational Ideas. The section links point to the parts of the book it draws on.