Posterior probability is the conditional distribution of an unknown quantity after observing data. For model parameters θ and training data X, it is p(θ|X): Bayes’ theorem combines the likelihood p(X|θ), which reflects the data under a model, with the prior p(θ), which expresses beliefs before seeing X. Up to a normalization constant, p(θ|X) is proportional to p(X|θ)p(θ). section 6.6.1 section 13.11
The posterior does not discard either source of information. With no data, parameter estimates are determined by the prior. With little data, a strong prior can substantially influence the result; as data grows, the likelihood increasingly dominates, and Bayesian estimates approach maximum-likelihood estimates. This makes posterior-based estimation useful when data is scarce but credible prior knowledge is available. section 6.6.3 section 13.3
Two closely related uses distinguish a distribution from a point estimate:
- Fully Bayesian estimation retains the posterior distribution, representing uncertainty about parameters. For a Gaussian likelihood with unknown mean and known variance, a Gaussian prior yields a Gaussian posterior. section 13.3 section 13.9
- Maximum a posteriori (MAP) estimation selects the parameter value at which the posterior is largest. Because the evidence p(X) is independent of θ, this maximizes p(X|θ)p(θ), making MAP equivalent to likelihood optimization with a prior-derived regularizer. section 6.6.3 section 13.6.1
- Conjugate priors preserve a convenient distribution family after updating: gamma priors for Gaussian precision produce gamma posteriors, while normal-gamma priors for unknown Gaussian mean and precision produce normal-gamma posteriors. section 13.6.1 section 13.7.2
Gaussian parameter update
Suppose observations are Gaussian with known variance but unknown mean μ. Start with a normal prior centered at μ₀, then combine it with the sample mean and sample count n to obtain a normal posterior with updated mean μₙ and variance σₙ. A PyTorch implementation returns this posterior as torch.Normal(μₙ, σₙ), so later code can use the full distribution rather than only one estimate. section 13.3
Posterior responsibilities in a Gaussian mixture
Posterior probability can also concern a latent class rather than a parameter. In a Gaussian mixture model, γik = p(k|x(i)) is the posterior probability that data point i belongs to component k. During iterative fitting, these responsibilities weight each point’s contribution to component means and covariances; for classification, choose the class with the largest p(Z=k|X=x). section 6.9.4 section 6.9.3
In variational autoencoders, the desired posterior p(z|x) links an input to latent variables. The encoder emits an approximation q(z|x), and training aims to make it close to the unknown true posterior using KL divergence. Directly maximizing the posterior is impractical because its normalization would require recomputing latent vectors across the full data set as network weights change, motivating ELBO-based methods. section 14.7.3 section 14.7.5
