liveBook · Manning

Evidence lower bound

Also called ELBO

The evidence lower bound (ELBO) is a variational objective that lower-bounds a model’s log marginal likelihood and is optimized in variational inference.

How 5 Manning books explain it

Bayesian Optimization in Action

The evidence lower bound is a tractable variational objective that lower-bounds the marginal log likelihood and is optimized in approximate Bayesian inference.

How it is used
Optimizes a scalable variational Gaussian process when full-data marginal likelihood computation is unstable or expensive.
How deep it goes
Defines the KL-based decomposition, explains batch estimation, and implements VGP optimization in GPyTorch.
Examples
Uses a KL-divergence decomposition, VGP inducing points, GPyTorch's VariationalELBO, and batch training code.
What you can do after
Apply it via GPyTorch to train a variational Gaussian process on medium-to-large datasets with mini-batch updates.

Read how Bayesian Optimization in Action explains evidence lower bound →

Machine Learning Algorithms in Depth

The evidence lower bound is a lower bound on the log marginal likelihood that serves as an optimization objective in variational inference.

How it is used
Serves as the objective for learning variational parameters that approximate latent-variable posteriors.
How deep it goes
Derives the bound using Jensen’s inequality, interprets its terms, and computes it in mean-field applications.
Examples
Uses binary-image denoising and latent Dirichlet allocation, including an ELBO-versus-iteration plot.
What you can do after
Build it from scratch to guide mean-field updates and track optimization in image denoising or topic inference.

Read how Machine Learning Algorithms in Depth explains evidence lower bound →

Grokking Bayes

The evidence lower bound (ELBO) is a variational inference objective that lower-bounds the log marginal likelihood and is maximized to fit an approximating distribution.

How it is used
Guides variational inference by scoring approximations and monitoring gradient-based optimization.
How deep it goes
Derives a computable KL-based objective and explains Monte Carlo estimation, gradient updates, and convergence checks.
Examples
Uses a Beta target, a squashed Normal approximator, gradient iterations, ELBO histories, and multimodal approximation failures.
What you can do after
Diagnose variational optimization with PyMC fitting histories and Matplotlib plots, while recognizing approximation failures.

Read how Grokking Bayes explains evidence lower bound →

Math and Architectures of Deep Learning

The variational objective optimized in a VAE, combining reconstruction quality with a regularization term on the latent distribution.

How it is used
Optimizes variational autoencoders when direct posterior normalization is computationally intractable.
How deep it goes
Derives the lower-bound identity, interprets entropy and expected log joint, and reaches the PyTorch loss decomposition.
Examples
Uses MNIST reconstructions and two-dimensional latent-space plots to contrast autoencoder and VAE objectives.
What you can do after
Apply it via PyTorch by combining binary cross-entropy reconstruction loss with KL regularization.

Read how Math and Architectures of Deep Learning explains evidence lower bound →

Causal AI

The evidence lower bound is a variational objective that lower-bounds the expected log-likelihood of observed data.

How it is used
Optimizes variational autoencoder training by balancing reconstruction quality with posterior approximation.
How deep it goes
Explains indirect optimization of reconstruction error and KL divergence, then connects the objective to batch-based training and implementation.
Examples
Uses Pyro SVI configuration with Adam and Trace_ELBO, plus batch training and test-loss evaluation.
What you can do after
Builds VAE training with Pyro’s SVI and Trace_ELBO while monitoring training and test loss.

Read how Causal AI explains evidence lower bound →

What to know first

Related concepts

Questions about evidence lower bound

What is evidence lower bound?

The evidence lower bound (ELBO) is a variational objective that lower-bounds a model’s log marginal likelihood and is optimized in variational inference.

Which Manning books explain evidence lower bound?

Bayesian Optimization in Action, Machine Learning Algorithms in Depth, Grokking Bayes, Math and Architectures of Deep Learning and Causal AI.

What should I know before learning evidence lower bound?

Manning books that teach evidence lower bound usually introduce prior probability, adam, approximate distribution and marginal likelihood first.

What is the difference between evidence lower bound and Kullback–Leibler divergence?

The evidence lower bound (ELBO) is a variational objective that lower-bounds a model’s log marginal likelihood and is optimized in variational inference. Kullback–Leibler divergence is a nonnegative, generally asymmetric measure of the difference between probability distributions, defined as the expected log probability ratio under one distribution.

What is evidence lower bound a kind of?

Evidence lower bound is a kind of objective function. An objective function is a mathematical function that assigns values to candidate solutions or parameter settings, which an optimization procedure seeks to minimize or maximize.

What is evidence lower bound also called?

Evidence lower bound is also called ELBO.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.