How deeply does Causal AI cover approximate distribution?

Explains parameter optimization, KL divergence, ELBO, and minibatch SVI, but defers detailed distribution-family choices.

Approximate distribution

As explained in Causal AI

A variational distribution is a parameterized probability distribution chosen to approximate a target distribution in variational inference.

An approximate distribution is a parameterized probability distribution designed to resemble a target distribution when direct sampling or likelihood evaluation is impractical. In variational inference, an optimized stochastic process generates samples from this approximation. section 5.2.3 section 2.2.5

The approximation becomes an operational component of inference:

A guide function produces samples from the approximate distribution, while its parameters are adjusted with gradient-based optimization. section 2.2.5
Training shifts the approximation toward the target by minimizing their difference, commonly through KL divergence and the ELBO objective. section 5.2.3
In stochastic variational inference, randomly selected data batches reduce memory use and support larger datasets. section 5.2.3

Latent-image inference with a VAE

An encoder defines Q(Z|X, is-handwritten, digit), the approximate distribution over latent Z for an image and its observed attributes. Sampling Z from this distribution lets the decoder parameterize the image distribution. In Pyro, the encoder serves as the guide, and SVI combines it with Trace_ELBO for training. section 5.2.3

How Causal AI covers it

liveBook manning.com

Read Causal AI on liveBook Get the book at manning.com

Also explained in

liveBook manning.com

Machine Learning Algorithms in DepthAn approximate posterior distribution is a tractable distribution used to approximate the conditional distribution of latent variables given observed data.
liveBook manning.com

Bayesian Optimization in ActionA variational distribution is an optimized probability distribution that approximates an intractable posterior distribution in variational inference.

Compare how 3 books explain approximate distribution →

Related concepts

variational inferenceused by · Variational inference is an approximate inference technique that optimizes a tractable family of probability distributions to approximate a target distribution, often a posterior.
amortized inferenceused by · Amortized inference is a technique that learns a reusable mapping from observations to approximate posterior distributions or their parameters.
latent Dirichlet allocationused by · Latent Dirichlet allocation is a generative probabilistic topic model representing documents as mixtures of latent topics and topics as distributions over words.
posterior probabilitycontrasts with · A posterior probability or distribution expresses the probability assigned to an event, hypothesis, or unknown quantity conditional on observed evidence and prior information.
variational Gaussian processhas part · A variational Gaussian process is a scalable approximation to a Gaussian process whose posterior is represented through variational inference over a finite set of inducing points.
guide functionproduced by · A guide function is a Pyro stochastic function that defines the variational approximating process used for inference.

This explainer was generated by Manning's knowledge graph from the text of Causal AI. The section links point to the parts of the book it draws on.