q(z|x) is the encoder-defined approximate posterior over latent vectors z for an input x. Because the true posterior p(z|x) is unknown, the encoder learns a tractable approximation, commonly a Gaussian distribution whose parameters are emitted separately for each input. section 14.7.2 section 14.7.6
The distribution connects an input to the VAE's latent representation and training objective:
- The encoder maps x to distribution parameters, such as the mean μ(x) and covariance Σ(x), rather than directly producing one deterministic latent vector. section 14.7.2
- The model samples z from q(z|x), then supplies z to the decoder to produce a reconstruction x̃. section 14.7.2
- Training balances reconstruction quality with KL divergence between q(z|x) and a fixed target prior, typically a zero-mean, unit-covariance Gaussian. section 14.7.2
- Across inputs, the KL term encourages encoder-generated latent distributions to align with regions where the target prior has high probability, supporting a compact latent space. section 14.7.2
- Minimizing the divergence between the unknown posterior and q(z|x) is equivalent to maximizing the ELBO. section 14.7.6
Gaussian encoder path
For one input x, the encoder emits μ(x) and Σ(x), defining q(z|x) = 𝒩(z; μ(x), Σ(x)). A latent sample z from this distribution goes to the decoder, which emits x̃. The training step evaluates reconstruction loss for x̃ and KL divergence for q(z|x), then minimizes their weighted sum. section 14.7.2
Figure 14.8 between the encoder-generated distribution (𝒩(
; 
(
), Σ()) ) and the target distribution p() (here p() ≡ 𝒩(; 
, I)) or, equivalently, low KL divergence between them.
|
(a) Low KL divergence loss (high q( |
|
(b) High KL divergence loss (high q( |
Compare low and high KL divergence by viewing where the encoder-generated Gaussian and the fixed target Gaussian place probability mass. section 14.7.2
