Normal-Gamma Distribution
Overview
The normal-gamma distribution is a probability distribution used for two random variables, typically representing the mean (μ) and precision (λ) of a Gaussian distribution. It is characterized by four parameters: μ′, λ′, α′, and β′. This distribution serves as the conjugate prior for a Gaussian distribution with unknown mean and precision, making it particularly useful in Bayesian estimation. When estimating both the mean and precision of a Gaussian likelihood, the prior is modeled as a normal-gamma distribution, and the resulting posterior is also a normal-gamma distribution.
PyTorch Implementation
The normal-gamma distribution is not natively implemented in PyTorch. However, a basic version can be constructed using the following code:
[Listing 13.3 PyTorch code for a normal-gamma distribution](https://livebook.manning.com/book/math-and-architectures-of-deep-learning/chapter-13/123)
import torch
class NormalGamma(): #1
def __init__(self, mu_, lambda_, alpha_, beta_):
self.mu_ = mu_
self.lambda_ = lambda_
self.alpha_ = alpha_
self.beta_ = beta_
@property
def mean(self):
return self.mu_, self.alpha_/ self.beta_
@property
def mode(self):
return self.mu_, (self.alpha_-0.5)/ self.beta_
#1 Since PyTorch doesnât implement normal-gamma distribution, we implement a bare-bones version.
This code defines a simple class NormalGamma that initializes with the parameters μ′, λ′, α′, and β′. It provides properties to compute the mean and mode of the distribution. The mean is given by the pair (μ, α′/β′), and the mode is given by (μ, (α′-0.5)/β′). This implementation is a straightforward way to work with the normal-gamma distribution in PyTorch, given its absence in the library's standard offerings.