Evidence lower bound
The evidence lower bound is a variational objective that lower-bounds a model’s log marginal likelihood and is optimized to approximate the posterior distribution.
Bayesian Optimization in Action
The evidence lower bound is a tractable variational objective that lower-bounds the marginal log likelihood and is optimized in approximate Bayesian inference.
- How it is used
- Optimizes a scalable variational Gaussian process when full-data marginal likelihood computation is unstable or expensive.
- How deep it goes
- Defines the KL-based decomposition, explains batch estimation, and implements VGP optimization in GPyTorch.
- Examples
- Uses a KL-divergence decomposition, VGP inducing points, GPyTorch's VariationalELBO, and batch training code.
- What you can do after
- Apply it via GPyTorch to train a variational Gaussian process on medium-to-large datasets with mini-batch updates.
Read how Bayesian Optimization in Action explains evidence lower bound →
Machine Learning Algorithms in Depth
The evidence lower bound is a lower bound on the log marginal likelihood that serves as an optimization objective in variational inference.
- How it is used
- Serves as the objective for learning variational parameters that approximate latent-variable posteriors.
- How deep it goes
- Derives the bound using Jensen’s inequality, interprets its terms, and computes it in mean-field applications.
- Examples
- Uses binary-image denoising and latent Dirichlet allocation, including an ELBO-versus-iteration plot.
- What you can do after
- Build it from scratch to guide mean-field updates and track optimization in image denoising or topic inference.
Read how Machine Learning Algorithms in Depth explains evidence lower bound →
Math and Architectures of Deep Learning
The variational objective optimized in a VAE, combining reconstruction quality with a regularization term on the latent distribution.
- How it is used
- Optimizes variational autoencoders when direct posterior normalization is computationally intractable.
- How deep it goes
- Derives the lower-bound identity, interprets entropy and expected log joint, and reaches the PyTorch loss decomposition.
- Examples
- Uses MNIST reconstructions and two-dimensional latent-space plots to contrast autoencoder and VAE objectives.
- What you can do after
- Apply it via PyTorch by combining binary cross-entropy reconstruction loss with KL regularization.
Read how Math and Architectures of Deep Learning explains evidence lower bound →
Causal AI
The evidence lower bound is a variational objective that lower-bounds the expected log-likelihood of observed data.
- How it is used
- Optimizes variational autoencoder training by balancing reconstruction quality with posterior approximation.
- How deep it goes
- Explains indirect optimization of reconstruction error and KL divergence, then connects the objective to batch-based training and implementation.
- Examples
- Uses Pyro SVI configuration with Adam and Trace_ELBO, plus batch training and test-loss evaluation.
- What you can do after
- Builds VAE training with Pyro’s SVI and Trace_ELBO while monitoring training and test loss.
Related concepts
- variational inferenceused by · Variational inference is an approximate inference technique that optimizes a tractable family of probability distributions to approximate a target distribution, often a posterior.
- variational autoencoderused by · A variational autoencoder is a generative neural network that learns a probabilistic latent representation of data and decodes samples from it to reconstruct or generate data.
- marginal likelihooduses · Marginal likelihood, or Bayesian evidence, is the probability or density of observed data under a model after marginalizing over unknown parameters or latent variables according to their prior distributions.
- objective functionkind of · An objective function assigns a value to candidate solutions, outputs, or parameter settings according to a criterion that an optimization procedure seeks to minimize or maximize.
- Kullback–Leibler divergencealternative to · Kullback–Leibler divergence is a directed measure of discrepancy between probability distributions, defined as the expected log ratio of their probabilities under one distribution.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.