Conjugacy
Conjugacy is the property whereby a prior distribution and a specified likelihood yield a posterior distribution in the same family as the prior.
Machine Learning Algorithms in Depth
A conjugate prior is a prior distribution whose posterior belongs to the same distributional family after updating with data.
- How it is used
- Simplifies Bayesian updating by keeping the posterior in the prior’s distributional family.
- How deep it goes
- Works through Beta posterior count updates for coin flips but leaves the Dirichlet–multinomial posterior as an exercise.
- Examples
- Uses coin-flip counts, Bernoulli naive Bayes, and a Dirichlet–multinomial posterior exercise.
- What you can do after
- Build it from scratch for coin flips by updating Beta prior counts with observed heads and tails.
Read how Machine Learning Algorithms in Depth explains conjugacy →
Math and Architectures of Deep Learning
A prior distribution chosen so that, after Bayes' rule is applied with a specified likelihood family, the posterior stays in the same distribution family.
- How it is used
- Uses distribution-family preservation to simplify Bayesian parameter estimation and prediction.
- How deep it goes
- Derives Gaussian and gamma updates, extends them to multivariate cases, and stops short of general nonconjugate inference.
- Examples
- Uses Gaussian mean estimation, gamma precision estimation, normal-gamma joint estimation, Wishart multivariate estimation, and PyTorch code.
- What you can do after
- Apply conjugate updating and compute a gamma posterior for unknown variance with known mean in PyTorch.
Read how Math and Architectures of Deep Learning explains conjugacy →
Causal AI
A conjugate prior is a prior distribution whose posterior distribution has the same mathematical form as the prior when combined with a specified likelihood.
- How it is used
- Uses Bayesian parameter learning for categorical conditional probabilities as an alternative to maximum-likelihood count proportions.
- How deep it goes
- Defines prior-posterior form matching and explains Dirichlet-categorical updating through counts and prior parameters, without sampling or variational inference.
- Examples
- Uses a Dirichlet prior to smooth commute-mode probabilities, including an extreme 100% estimate for self-employed people in small towns who drive.
- What you can do after
- Apply via pgmpy’s BayesianEstimator with a Dirichlet prior and pseudo_counts=1 to estimate categorical model parameters.
Related concepts
- prior probabilitykind of · A prior probability distribution represents beliefs or assumptions about an unknown quantity before relevant observations or evidence are incorporated.
- Dirichlet distributionincludes · A Dirichlet distribution is a continuous probability distribution over vectors of nonnegative components that sum to one, parameterized by positive concentration parameters.
- bayesian statisticsuses · Bayesian statistics is a framework for statistical inference that combines prior probability distributions with observed-data likelihoods to update probabilities over hypotheses or unknown quantities.
- posterior probabilityproduces · A posterior probability or distribution expresses the probability assigned to an event, hypothesis, or unknown quantity conditional on observed evidence and prior information.
- gamma distributionincludes · The gamma distribution is a continuous probability distribution over positive values, characterized by shape and scale or rate parameters.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.