Suppose you want to estimate the proportion of coworkers who prefer tea to coffee. You could test a list of possible proportions, but rounding to that list leaves out values between them. A continuous prior can represent every proportion from 0 to 1. The challenge is updating it without a difficult calculation over all those possibilities. section 3.0 section 3.1
Conjugacy offers a shortcut. A prior is conjugate to a likelihood when the posterior belongs to the same distribution family as the prior. Data changes the distribution’s parameters, but not its family, so a known rule gives the posterior directly. This is a closed-form Bayesian update rather than an approximation. section 3.6 section 3.18
Updating a tea-preference estimate
Let p be the proportion who prefer tea. Start with Beta(1, 1), a flat prior over values from 0 to 1. For a beta prior with parameters α and β, a binomial likelihood with k successes in n trials gives a Beta(α + k, β + n − k) posterior. If 3 of 5 coworkers prefer tea, the posterior is Beta(4, 3). It peaks at 0.6, while still representing uncertainty across possible values of p. section 3.7
prior_a = 1 #A prior_b = 1 #A k = 3 #B n = 5 #B post_a = prior_a + k #C post_b = prior_b + (n - k) #C
The short calculation separates the prior parameters, observed counts, and posterior parameters; the update requires addition rather than numerical integration. section 3.7
Conjugacy does not require a flat prior. Beta distributions can express different beliefs about a proportion: some favor the extremes, while others favor values near the middle. When the prior and data agree, the posterior can become more concentrated; when they disagree, it reflects both. Choosing the prior therefore affects the answer, especially when data are limited. section 3.9 section 3.18
The idea also applies to real-valued quantities. A normal prior paired with a normal likelihood yields a normal posterior, as when estimating a house-price model’s slope. That convenience has a cost: a normal prior gives some probability to negative slopes, even when a negative price-per-area relationship is implausible. If representing a non-negative slope matters more than retaining conjugacy, a different prior is needed; Bayesian updating remains valid, but the posterior may no longer have an easy closed-form calculation. section 3.13 section 5.2




