liveBook · Manning

Variational inference

Variational inference is an approximate inference technique that optimizes a tractable family of probability distributions to approximate a target distribution, often a posterior.

How 3 Manning books explain it

Causal AI

Variational inference approximates a target probability distribution by optimizing the parameters of a tractable distribution or stochastic process.

How it is used
Performs probabilistic inference for latent-variable model training and counterfactual abduction.
How deep it goes
Defines guide-based approximation and works through conditioning, optimization, and extraction of fitted distributions in Pyro.
Examples
Uses image reconstruction and a femur-length intervention conditioned on observed femur length and height.
What you can do after
Apply it via Pyro to fit a guide for latent variables and use its inferred distribution in counterfactual prediction.

Read how Causal AI explains variational inference →

Grokking Bayes

Variational inference approximates an intractable posterior distribution by optimizing within a tractable family of distributions.

How it is used
Approximates Bayesian posteriors when sampling is too slow for large datasets or complex models.
How deep it goes
Develops candidate-family optimization through KL divergence, ELBO, reparameterization and PyMC fitting, then examines approximation failures.
Examples
Uses Bayesian linear regression posteriors and Bayesian neural network predictions to expose both speed gains and fitting problems.
What you can do after
Apply it via PyMC’s `pm.fit()`, sample the fitted approximation and inspect whether its predictions and uncertainty are credible.

Read how Grokking Bayes explains variational inference →

Machine Learning Algorithms in Depth

Accurate variational inference uses richer variational models than fully factorized approximations to improve posterior approximation quality.

How it is used
Approximates difficult Bayesian posteriors by optimizing a tractable distribution instead of approximating them through MCMC samples.
How deep it goes
Derives an ELBO and implements mean-field variational Bayes for LDA, but does not specify a richer-family update rule.
Examples
Uses Gaussian-mixture KL fits and a from-scratch LDA implementation with ELBO plots and topic word clouds.
What you can do after
Build it from scratch for mean-field LDA and evaluate approximation behavior through the ELBO and KL direction.

Read how Machine Learning Algorithms in Depth explains variational inference →

Also covered in

What to know first

Related concepts

Questions about variational inference

What is variational inference?

Variational inference is an approximate inference technique that optimizes a tractable family of probability distributions to approximate a target distribution, often a posterior.

Which Manning books explain variational inference?

Causal AI, Grokking Bayes and Machine Learning Algorithms in Depth.

What should I know before learning variational inference?

Manning books that teach variational inference usually introduce markov chain monte carlo, posterior probability, prior probability and probability density first.

What is the difference between variational inference and markov chain monte carlo?

Variational inference is an approximate inference technique that optimizes a tractable family of probability distributions to approximate a target distribution, often a posterior. Markov chain Monte Carlo is a family of sampling algorithms that uses a Markov chain to generate samples from a target probability distribution.

What kinds of variational inference are there?

Kinds of variational inference include stochastic variational inference.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.