The evidence lower bound (ELBO) is the score that makes variational inference practical. Instead of calculating an inaccessible posterior directly, it evaluates how well a manageable approximating distribution q fits that posterior. Maximizing the ELBO reduces the KL divergence between q and the true posterior. section 6.15 section 6.4
Use the ELBO as both an optimization target and a progress check:
- Estimate the ELBO using values from the unnormalized posterior, q, and samples from q; Monte Carlo weighted sums approximate the required integral. section 6.4
- Adjust the parameters of q with gradient descent so that the ELBO increases and the approximation moves closer to the target posterior. section 6.6
- Plot the ELBO during fitting; a flattening curve suggests that optimization has converged, while an early cutoff may require more steps. section 6.10
Fitting a Normal approximation to a Beta target
Imagine a target Beta distribution on values from 0 to 1 and a Normal-based approximator q that starts concentrated near 0. Gradient descent changes q’s mean and spread: the ELBO rises as q moves toward the target, then adjusts its spread for a closer fit. The final approximation can still miss important structure when the target is multimodal or has broader variance. section 6.6 section 6.13
import matplotlib.pyplot as plt plt.plot(vi_approx.hist) plt.show()
Plotting vi_approx.hist with Matplotlib turns the ELBO history into a direct visual convergence check. section 6.10




