The gamma distribution models a positive continuous quantity with shape α and rate β. In Bayesian estimation, it serves as a prior and posterior for Gaussian precision λ, where λ = 1/σ². section 13.6 section 13.11
Its role in Gaussian parameter estimation follows from conjugacy:
- For known mean and unknown variance, assign λ a gamma prior: p(λ) = γ(λ; α₀, β₀). section 13.6.1
- Multiplying the Gaussian likelihood by the gamma prior produces another gamma posterior, p(λ|X) = γ(λ; αₙ, βₙ). section 13.6 section 13.6.1
- The posterior parameters incorporate both prior information and data; increasing sample size shifts estimation toward the likelihood. section 13.6.1
- Maximizing the posterior gives the MAP precision estimate λMAP = (αₙ − 1)/βₙ. section 13.6.1
Known mean, unknown variance
Given observations X from a Gaussian with known mean, compute the sample standard deviation and update α₀, β₀ to αₙ = n/2 + α₀ and βₙ = nσMLE²/2 + β₀. The resulting gamma posterior estimates precision and therefore variance. section 13.6.1
Listing 13.2 PyTorch- Bayesian estimation with unknown variance, known mean
import torch
def inference_unknown_variance(X, prior_dist):
sigma_mle = torch.std(X)
n = X.shape[0]
alpha_0 = prior_dist.concentration
beta_0 = prior_dist.rate #1
alpha_n = n / 2 + alpha_0
beta_n = n / 2 * sigma_mle ** 2 + beta_0 #2
posterior_dist = torch.Gamma(alpha_n, beta_n)
return posterior_dist
A PyTorch implementation constructs the gamma posterior from the prior concentration and rate, sample size, and sample variance. section 13.6.1
