ExactMarginalLogLikelihood

As explained in Bayesian Optimization in Action

ExactMarginalLogLikelihood is a GPyTorch module that computes the exact marginal log likelihood of training data under a Gaussian process model.

ExactMarginalLogLikelihood measures how well a Gaussian process explains its training data under specified model hyperparameters. Comparing likelihoods lets you choose parameters, such as a constant mean: a positive value can fit exclusively positive observations better than zero or a negative value. section 3.3.2

A typical GPyTorch optimization loop:

• Construct it with the likelihood and GP model: gpytorch.mlls.ExactMarginalLogLikelihood(likelihood, model). section 3.3.2
• Evaluate the model output, negate the marginal log likelihood as the loss, backpropagate, and update the parameter with gradient descent. section 3.3.2
• Track loss and parameter values until they converge, then switch the model and likelihood to evaluation mode. section 3.3.2

Selecting a constant mean

With lengthscale and noise fixed, optimize only the model's constant mean. A GP whose mean reverts to 2 achieves higher likelihood than one reverting to 0, indicating that 2 better explains the training observations. section 3.3.2

Figure 3.6 GP predictions, given two different constant mean functions. The value 2 gives a higher likelihood value than the value 0, indicating the former mean function is a better fit than the latter.

Compare GP predictions using constant mean values 0 and 2; the figure visualizes why the value 2 receives higher likelihood. section 3.3.2

How Bayesian Optimization in Action covers it

How deep it goes

Defines the data-fit role and demonstrates construction, negated-loss optimization, convergence tracking, and evaluation mode.

Examples

Compares constant mean values 0 and 2 while optimizing a GPyTorch GP with fixed lengthscale and noise.

What it assumes

Assumes a GP model, likelihood, training data, and gradient-descent basics while motivating likelihood through mean-function comparisons.

What you can do after

Apply it via GPyTorch to optimize a GP parameter with gradient descent against negative marginal log likelihood.

How it connects

Connects Gaussian processes, likelihood, mean functions, hyperparameters, gradient descent, and Gaussian likelihoods.

liveBookmanning.com

Read Bayesian Optimization in Action on liveBook Get the book at manning.com

Related concepts

• Gaussian processuses · A Gaussian process is a probability distribution over functions in which every finite collection of function values has a joint Gaussian distribution.
• PairwiseLaplaceMarginalLogLikelihoodalternative to · PairwiseLaplaceMarginalLogLikelihood is a BoTorch marginal-log-likelihood implementation for Gaussian-process models trained on pairwise comparisons.

This explainer was generated by Manning's knowledge graph from the text of Bayesian Optimization in Action. The section links point to the parts of the book it draws on.