liveBook · Manning

Unnormalized posterior

As explained in Grokking Bayes

An unnormalized posterior is a probability mass or density proportional to the posterior before normalization.

Suppose you are deciding whether it will rain. You have an initial belief about rain, then see a dark cloud. An unnormalized posterior combines those two pieces of information for each possibility: multiply its prior probability by the likelihood of seeing the cloud if that possibility were true. The products show how the possibilities compare, but they are not yet probabilities that sum to 1. section 2.2 section 2.4

For a hypothesis H and observed data D, the unnormalized posterior is Pr(D | H) × Pr(H). Calculate it for every competing hypothesis, add the products, then divide each product by that sum. The resulting posterior probabilities sum to 1. The sum used for division is called the evidence, Pr(D). section 2.2

Three hypotheses by hand

Start with prior probabilities 0.4, 0.2, and 0.4. If the corresponding likelihoods are 0.3, 0.2, and 0.1, the unnormalized posteriors are 0.12, 0.04, and 0.04. Their sum is 0.20. Dividing each by 0.20 gives posterior probabilities 0.6, 0.2, and 0.2. The second and third hypotheses finish tied: the third began with twice the prior probability, but the second explains the data twice as well. section 2.3

The same calculation scales to a grid of possible values. For the proportion of people who prefer tea, eleven values from 0 to 1 receive equal priors. After a survey finds that three of five people prefer tea, a binomial likelihood is calculated for each value. Multiplying the arrays of priors and likelihoods produces the unnormalized posterior; dividing by its sum produces the posterior. The result peaks around 0.6, reflecting the observed three out of five. section 2.11

unnormalized_posterior = prior * likelihood
posterior = (
    unnormalized_posterior
    / unnormalized_posterior.sum()
)

These two Python lines perform the final steps: elementwise prior-times-likelihood multiplication, followed by division by the sum of those products. section 2.11

A zero prior or zero likelihood makes that hypothesis’s unnormalized posterior—and therefore its posterior probability—zero. For larger or harder problems, explicitly calculating a grid may become slow or numerically unstable. Sampling methods can instead approximate a posterior even when only its unnormalized form is known. section 2.6 section 3.1 section 5.14

How Grokking Bayes covers it

How deep it goes
Works through multiplication and normalization by hand and in NumPy; leaves large-grid limitations and sampling as later extensions.
Examples
Uses a three-hypothesis calculation, dark-cloud weather update, and tea-preference survey with eleven candidate proportions.
What it assumes
Introduces hypotheses, priors, and likelihoods before the calculation; uses multiplication, addition, and division in its hand example.
What you can do after
Build it from scratch for a small hypothesis set, then apply it with NumPy to normalize a grid of posterior weights.
How it connects
Links Bayes’ theorem, prior probability, likelihood, evidence, posterior probability, and Monte Carlo sampling.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Grokking Bayes. The section links point to the parts of the book it draws on.