Normal-Gamma Distribution

Overview

The normal-gamma distribution is a probability distribution used for two random variables, typically representing the mean (μ) and precision (λ) of a Gaussian distribution. It is characterized by four parameters: μ′, λ′, α′, and β′. This distribution serves as the conjugate prior for a Gaussian distribution with unknown mean and precision, making it particularly useful in Bayesian estimation. When estimating both the mean and precision of a Gaussian likelihood, the prior is modeled as a normal-gamma distribution, and the resulting posterior is also a normal-gamma distribution.

PyTorch Implementation

The normal-gamma distribution is not natively implemented in PyTorch. However, a basic version can be constructed using the following code:

[Listing 13.3 PyTorch code for a normal-gamma distribution](https://livebook.manning.com/book/math-and-architectures-of-deep-learning/chapter-13/123)

import torch

class NormalGamma():                            #1


    def __init__(self, mu_, lambda_, alpha_, beta_):
        self.mu_ = mu_
        self.lambda_ = lambda_
        self.alpha_ = alpha_
        self.beta_ = beta_

    @property
    def mean(self):
        return self.mu_, self.alpha_/ self.beta_

    @property
    def mode(self):
        return self.mu_, (self.alpha_-0.5)/ self.beta_
#1 Since PyTorch doesn’t implement normal-gamma distribution, we implement a bare-bones version.

This code defines a simple class NormalGamma that initializes with the parameters μ′, λ′, α′, and β′. It provides properties to compute the mean and mode of the distribution. The mean is given by the pair (μ, α′/β′), and the mode is given by (μ, (α′-0.5)/β′). This implementation is a straightforward way to work with the normal-gamma distribution in PyTorch, given its absence in the library's standard offerings.