How deeply does Bayesian Optimization in Action cover approximate distribution?

Defines implementation components and training roles, but stops short of deriving the variational distribution mathematically.

Approximate distribution

As explained in Bayesian Optimization in Action

A variational distribution is an optimized probability distribution that approximates an intractable posterior distribution in variational inference.

A variational distribution defines a probability distribution over a VGP’s inducing points. Optimization adjusts it so the variational model resembles a GP trained on the full dataset. section 12.2.3

In a GPyTorch variational Gaussian process, the distribution:

Uses CholeskyVariationalDistribution, initialized with the number of inducing points. section 12.2.3
Works with a variational strategy that updates the distribution and produces predictions from inducing points. section 12.2.3
Is optimized with the VGP’s ELBO during mini-batch gradient descent. section 12.2.3

VGP construction

Create the distribution with the number of inducing points, pass it to VariationalStrategy with learn_inducing_locations=True, and train the resulting model with VariationalELBO. section 12.2.3

Table 12.3 Necessary components when implementing a GP vs. a VGP in GPyTorch. A VGP requires a mean and covariance module like a GP but additionally needs a variational distribution and a variational strategy. (view table figure)

Components	GP	VGP
Mean module	Yes	Yes
Covariance module	Yes	Yes
Variational distribution	No	Yes
Variational strategy	No	Yes

Compare the components required by a standard GP and a VGP: the VGP adds a variational distribution and variational strategy. section 12.2.3

How Bayesian Optimization in Action covers it

Cover of Bayesian Optimization in Action

liveBook manning.com

Read Bayesian Optimization in Action on liveBook Get the book at manning.com

Also explained in

liveBook manning.com

Causal AIA variational distribution is a parameterized probability distribution chosen to approximate a target distribution in variational inference.
liveBook manning.com

Machine Learning Algorithms in DepthAn approximate posterior distribution is a tractable distribution used to approximate the conditional distribution of latent variables given observed data.

Compare how 3 books explain approximate distribution →

Related concepts

variational inferenceused by · Variational inference is an approximate inference technique that optimizes a tractable family of probability distributions to approximate a target distribution, often a posterior.
amortized inferenceused by · Amortized inference is a technique that learns a reusable mapping from observations to approximate posterior distributions or their parameters.
latent Dirichlet allocationused by · Latent Dirichlet allocation is a generative probabilistic topic model representing documents as mixtures of latent topics and topics as distributions over words.
posterior probabilitycontrasts with · A posterior probability or distribution expresses the probability assigned to an event, hypothesis, or unknown quantity conditional on observed evidence and prior information.
variational Gaussian processhas part · A variational Gaussian process is a scalable approximation to a Gaussian process whose posterior is represented through variational inference over a finite set of inducing points.
guide functionproduced by · A guide function is a Pyro stochastic function that defines the variational approximating process used for inference.

This explainer was generated by Manning's knowledge graph from the text of Bayesian Optimization in Action. The section links point to the parts of the book it draws on.