13 Fully Bayes model parameter estimation

Suppose we have a data set of interest: say, all images containing a cat. If we represent images as points in a high-dimensional feature space, our data set of interest forms a subspace of that feature space. Now we want to create an unsupervised model for our data set of interest. This means we want to identify a probability density function p() whose sample cloud (the set of points obtained by repeatedly sampling the probability distribution many times) largely overlaps our subspace of interest. Of course, we do not know the exact subspace of interest, but we have collected a set of samples X from the data set of interest: that is, the training data. We can use the point cloud for X as a surrogate for the unknown subspace of interest. Thus, we are essentially trying to identify a probability density function p() whose sample cloud, by and large, overlaps X.

Once we have the model p(), we can use it to generate more data samples; these will be computer-generated cat images. This is generative modeling. Also, given a new image , we can estimate the probability of it being an image of a cat by evaluating p().

13.1 Fully Bayes estimation: An informal introduction

Let’s recap Bayes’ theorem:

13.1.1 Parameter estimation and belief injection

13.2 MLE for Gaussian parameter values recap)

13.3 Fully Bayes parameter estimation: Gaussian, unknown mean, known precision

13.4 Small and large volumes of training data, and strong and weak priors

13.5 Conjugate priors

13.6 Fully Bayes parameter estimation: Gaussian, unknown precision, known mean

13.6.1 Estimating the precision parameter

13.7 Fully Bayes parameter estimation: Gaussian, unknown mean, unknown precision

13.7.1 Normal-gamma distribution

13.7.2 Estimating the mean and precision parameters

13.8 Example: Fully Bayesian inferencing

13.8.1 Maximum likelihood estimation

13.8.2 Bayesian inference

13.10.1 Wishart distribution