A conjugate prior is chosen for a likelihood family so Bayes’ rule produces a posterior in the same distribution family. This preserves a tractable form for updating beliefs and making predictions. section 13.5 section 13.11
For a Gaussian likelihood with known variance, a Gaussian prior over the mean gives a Gaussian posterior. For a Gaussian likelihood expressed through precision, a gamma prior gives a gamma posterior. With little data, the prior strongly influences the estimate; with more data, the likelihood dominates. section 13.5 section 13.6.1
Common Gaussian-model conjugate choices include:
- Gaussian prior for the mean when variance is known; the posterior remains Gaussian. section 13.5
- Gamma prior for precision, the inverse of variance; the posterior remains gamma. section 13.6.1
- Normal-gamma prior for jointly estimating mean and precision; the posterior remains normal-gamma. section 13.11
- Wishart prior for multivariate precision; the posterior remains Wishart. section 13.5
Unknown variance with known mean
Model precision λ with a gamma prior having parameters α₀ and β₀. After observing n data points, update to αₙ = n/2 + α₀ and βₙ = nσ²MLE/2 + β₀. The posterior is Gamma(αₙ, βₙ), and its MAP estimate is λMAP = (αₙ − 1)/βₙ. section 13.6.1
Listing 13.2 PyTorch- Bayesian estimation with unknown variance, known mean
import torch
def inference_unknown_variance(X, prior_dist):
sigma_mle = torch.std(X)
n = X.shape[0]
alpha_0 = prior_dist.concentration
beta_0 = prior_dist.rate #1
alpha_n = n / 2 + alpha_0
beta_n = n / 2 * sigma_mle ** 2 + beta_0 #2
posterior_dist = torch.Gamma(alpha_n, beta_n)
return posterior_dist
A PyTorch implementation computes updated gamma parameters from the sample size and maximum-likelihood variance, then returns the posterior distribution. section 13.6.1
