liveBook · Manning

Normal-gamma distribution

As explained in Math and Architectures of Deep Learning

A joint probability distribution over a mean parameter and a precision parameter, defined here as the product of a normal distribution and a gamma distribution.

The normal-gamma distribution jointly models a Gaussian mean parameter μ and a precision parameter λ. Its density is the product of a normal distribution and a gamma distribution, with parameters μ′, λ′, α′, and β′. section 13.7.1

For a Gaussian likelihood with unknown mean and precision, a normal-gamma prior produces another normal-gamma posterior. This conjugacy keeps Bayesian updating in the same distribution family and supports maximum-a-posteriori estimation. section 13.7 section 13.7.2

The parameter roles and computational updates are:

  • λ represents Gaussian precision, the inverse of variance; α and β parameterize the gamma component. section 13.7.2
  • Given sample count n, sample mean, sample standard deviation, and prior parameters, compute μₙ, λₙ, αₙ, and βₙ for the posterior. section 13.7.2
  • Read the posterior mode as an estimate of μ and precision, then convert precision to standard deviation using σ = √(1/λ). section 13.7.2

Estimating adult height

Prior information from a neighboring town sets μ₀ = 150, λ₀ = 100, α₀ = 10.5, and β₀ = 810. Applying the posterior update to height data yields MAP estimates μ = 149.98 and σ = 9.56, compared with maximum-likelihood estimates of 149.68 and 11.52. section 13.8.2

How Math and Architectures of Deep Learning covers it

How deep it goes
Derives conjugate posterior parameters, MAP estimates, and PyTorch counterparts for unknown Gaussian mean and precision.
Examples
Uses adult-height priors, posterior parameter calculations, MAP estimates, and PyTorch distribution classes.
What it assumes
Builds from Gaussian likelihoods with known-parameter cases, then combines normal and gamma priors for two unknown parameters.
What you can do after
Apply it via PyTorch to update a normal-gamma prior and recover MAP mean and standard-deviation estimates.
How it connects
Connects Gaussian likelihood, gamma distribution, conjugate prior, Bayesian inference, precision, and Wishart distribution.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Math and Architectures of Deep Learning. The section links point to the parts of the book it draws on.