liveBook · Manning

Conjugacy

As explained in Grokking Bayes

A conjugate prior is a prior distribution whose posterior belongs to the same distributional family as the prior for a specified likelihood.

Suppose you want to estimate the proportion of coworkers who prefer tea to coffee. You could test a list of possible proportions, but rounding to that list leaves out values between them. A continuous prior can represent every proportion from 0 to 1. The challenge is updating it without a difficult calculation over all those possibilities. section 3.0 section 3.1

Conjugacy offers a shortcut. A prior is conjugate to a likelihood when the posterior belongs to the same distribution family as the prior. Data changes the distribution’s parameters, but not its family, so a known rule gives the posterior directly. This is a closed-form Bayesian update rather than an approximation. section 3.6 section 3.18

Updating a tea-preference estimate

Let p be the proportion who prefer tea. Start with Beta(1, 1), a flat prior over values from 0 to 1. For a beta prior with parameters α and β, a binomial likelihood with k successes in n trials gives a Beta(α + k, β + n − k) posterior. If 3 of 5 coworkers prefer tea, the posterior is Beta(4, 3). It peaks at 0.6, while still representing uncertainty across possible values of p. section 3.7

prior_a = 1                #A
prior_b = 1                #A
 
k = 3                      #B
n = 5                      #B
 
post_a = prior_a + k       #C
post_b = prior_b + (n - k) #C

The short calculation separates the prior parameters, observed counts, and posterior parameters; the update requires addition rather than numerical integration. section 3.7

Conjugacy does not require a flat prior. Beta distributions can express different beliefs about a proportion: some favor the extremes, while others favor values near the middle. When the prior and data agree, the posterior can become more concentrated; when they disagree, it reflects both. Choosing the prior therefore affects the answer, especially when data are limited. section 3.9 section 3.18

The idea also applies to real-valued quantities. A normal prior paired with a normal likelihood yields a normal posterior, as when estimating a house-price model’s slope. That convenience has a cost: a normal prior gives some probability to negative slopes, even when a negative price-per-area relationship is implausible. If representing a non-negative slope matters more than retaining conjugacy, a different prior is needed; Bayesian updating remains valid, but the posterior may no longer have an easy closed-form calculation. section 3.13 section 5.2

How Grokking Bayes covers it

How deep it goes
Works through a beta–binomial parameter update and a normal–normal application, but defers broader conjugate-prior lookup tables.
Examples
Uses tea-versus-coffee preferences and a house-price regression slope to contrast convenient updates with prior-choice limits.
What it assumes
Builds on discrete Bayesian updating while introducing continuous priors and the beta–binomial update.
What you can do after
Apply it via a beta–binomial update to calculate a posterior and assess when conjugacy compromises a prior belief.
How it connects
Links Bayesian updating to beta–binomial models, normal–normal models, posterior uncertainty, and non-conjugate priors.

Also explained in

Compare how 5 books explain conjugacy →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Grokking Bayes. The section links point to the parts of the book it draws on.