Approximate distribution
An approximate posterior distribution is a tractable probability distribution selected or optimized to represent an intractable posterior distribution over unknown quantities.
Causal AI
A variational distribution is a parameterized probability distribution chosen to approximate a target distribution in variational inference.
- How it is used
- Uses a parameterized guide to make probabilistic inference tractable when direct access to the target distribution is unavailable.
- How deep it goes
- Explains parameter optimization, KL divergence, ELBO, and minibatch SVI, but defers detailed distribution-family choices.
- Examples
- Demonstrates latent-variable image reconstruction through a VAE encoder, decoder, guide, and Pyro SVI training loop.
- What you can do after
- Apply it via Pyro by defining or selecting a guide and optimizing it with SVI and Trace_ELBO.
Machine Learning Algorithms in Depth
An approximate posterior distribution is a tractable distribution used to approximate the conditional distribution of latent variables given observed data.
- How it is used
- Serves as a tractable stand-in for a target posterior in variational inference and in a mutual-information lower-bound procedure.
- How deep it goes
- Defines the approximation and develops KL-based optimization, with mean-field factorization and an ELBO objective.
- Examples
- Compares normal distributions while varying the approximate mean and contrasts forward-KL support coverage with reverse-KL mode selection.
- What you can do after
- Evaluate/understand how a tractable posterior approximation is selected and optimized, including the effect of KL direction.
Read how Machine Learning Algorithms in Depth explains approximate distribution →
Bayesian Optimization in Action
A variational distribution is an optimized probability distribution that approximates an intractable posterior distribution in variational inference.
- How it is used
- Uses an optimized distribution over inducing points to make variational Gaussian processes practical for large datasets.
- How deep it goes
- Defines implementation components and training roles, but stops short of deriving the variational distribution mathematically.
- Examples
- Uses CholeskyVariationalDistribution, VariationalStrategy, inducing-point learning, mini-batch training, and GP–VGP prediction comparisons.
- What you can do after
- Apply it via GPyTorch by constructing a variational distribution, connecting a strategy, and optimizing the ELBO.
Read how Bayesian Optimization in Action explains approximate distribution →
Related concepts
- variational inferenceused by · Variational inference is an approximate inference technique that optimizes a tractable family of probability distributions to approximate a target distribution, often a posterior.
- amortized inferenceused by · Amortized inference is a technique that learns a reusable mapping from observations to approximate posterior distributions or their parameters.
- latent Dirichlet allocationused by · Latent Dirichlet allocation is a generative probabilistic topic model representing documents as mixtures of latent topics and topics as distributions over words.
- posterior probabilitycontrasts with · A posterior probability or distribution expresses the probability assigned to an event, hypothesis, or unknown quantity conditional on observed evidence and prior information.
- variational Gaussian processhas part · A variational Gaussian process is a scalable approximation to a Gaussian process whose posterior is represented through variational inference over a finite set of inducing points.
- guide functionproduced by · A guide function is a Pyro stochastic function that defines the variational approximating process used for inference.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.