A prior is conjugate to a likelihood when updating it with observed data produces a posterior in the same distributional family. For parameters θ and data D, Bayes’ rule gives p(θ|D) = p(D|θ)p(θ)/p(D). Conjugacy makes the update closed-form: new data change the prior’s parameters without requiring a different form of posterior. The normalizing term p(D) can otherwise require an intractable integral. section 1.4
Coin-flip updates
Let θ be the probability of heads. A Beta prior can encode counts from earlier coin flips; after observing more heads and tails, update those counts. The posterior is again Beta, so the same kind of update can be repeated as data arrive. section 2.1.1 section 1.4
Conjugacy also offers a way to smooth count-based estimates in Bernoulli naive Bayes: Beta priors can be added for Bernoulli variables, and a Dirichlet prior for class probabilities. Another pairing to verify is a Dirichlet prior with a multinomial likelihood: compute the posterior and inspect how its shape changes with the resulting counts. section 5.5 section 8.4
