Fully Bayes parameter estimation combines the likelihood of observed data with a prior distribution to estimate unknown model parameters. In unsupervised generative modeling, the resulting probability density can approximate a data set such as cat images, generate new samples, and score new observations. section 13.0 section 13.3
The posterior is proportional to likelihood times prior. Unlike MLE, which uses only observed data, Bayesian estimation incorporates prior belief: limited data leaves the prior influential, while abundant data makes the posterior approach the MLE. section 13.1.1 section 13.3
Gaussian parameter-estimation cases include:
- Unknown mean with known precision: model the mean with a Gaussian prior and obtain a Gaussian posterior. section 13.3
- Known mean with unknown precision: use a gamma prior and obtain a gamma posterior through conjugacy. section 13.6.1
- Extend the treatment to unknown mean and variance, covering both univariate and multivariate cases. section 13.1.1
Posterior estimation in PyTorch
For an unknown Gaussian mean with known variance, compute the sample mean, combine it with the prior mean and scale, calculate posterior parameters, and return torch.Normal(mu_n, sigma_n). For unknown variance with known mean, update gamma concentration and rate and return torch.Gamma(alpha_n, beta_n). section 13.3 section 13.6.1
Listing 13.1 PyTorch- Bayesian estimation with unknown mean, known variance
import torch
def inference_unknown_mean(X, prior_dist, sigma_known):
mu_mle = X.mean()
n = X.shape[0]
mu_0 = prior_dist.mean
sigma_0 = prior_dist.scale #1
mu_n = mu_mle / (1 + sigma_known**2 / (n * sigma_0**2)) +
mu_0 / (1 + n * sigma_0**2 / sigma_known**2) #2
sigma_n = math.sqrt(
(sigma_0**2 * sigma_known**2) /
(n*sigma_0**2+sigma_known**2)) #3
posterior_dist = torch.Normal(mu_n, sigma_n)
return posterior_dist
Use the executable listing to connect the posterior formulas for an unknown mean to a concrete PyTorch implementation. section 13.3
