Variational inference
Variational inference is an approximate inference technique that optimizes a tractable family of probability distributions to approximate a target distribution, often a posterior.
Causal AI
Variational inference approximates a target probability distribution by optimizing the parameters of a tractable distribution or stochastic process.
- How it is used
- Performs probabilistic inference for latent-variable model training and counterfactual abduction.
- How deep it goes
- Defines guide-based approximation and works through conditioning, optimization, and extraction of fitted distributions in Pyro.
- Examples
- Uses image reconstruction and a femur-length intervention conditioned on observed femur length and height.
- What you can do after
- Apply it via Pyro to fit a guide for latent variables and use its inferred distribution in counterfactual prediction.
Machine Learning Algorithms in Depth
Accurate variational inference uses richer variational models than fully factorized approximations to improve posterior approximation quality.
- How it is used
- Approximates difficult Bayesian posteriors by optimizing a tractable distribution instead of approximating them through MCMC samples.
- How deep it goes
- Derives an ELBO and implements mean-field variational Bayes for LDA, but does not specify a richer-family update rule.
- Examples
- Uses Gaussian-mixture KL fits and a from-scratch LDA implementation with ELBO plots and topic word clouds.
- What you can do after
- Build it from scratch for mean-field LDA and evaluate approximation behavior through the ELBO and KL direction.
Read how Machine Learning Algorithms in Depth explains variational inference →
Related concepts
- stochastic variational inferenceincludes · Stochastic variational inference is a variational inference technique that optimizes an objective using stochastic estimates, often computed from randomly sampled data subsets.
- Kullback–Leibler divergenceuses · Kullback–Leibler divergence is a nonnegative, directed measure of the information lost when one probability distribution approximates another, computed as an expected log ratio under the first distribution.
- evidence lower bounduses · The evidence lower bound is a variational objective that lower-bounds a model’s log marginal likelihood and is optimized to approximate the posterior distribution.
- markov chain monte carloalternative to · Markov chain Monte Carlo is a family of sampling algorithms that generate samples using a Markov chain whose stationary or limiting distribution is a target probability distribution.
- approximate distributionuses · An approximate posterior distribution is a tractable probability distribution selected or optimized to represent an intractable posterior distribution over unknown quantities.
- optimizationuses · Optimization is the systematic selection or adjustment of values, actions, or system designs to improve a specified objective, often subject to constraints.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.