A powerful decoder can reconstruct inputs while ignoring an autoencoder’s latent code, a failure known as posterior collapse. The variational lossy autoencoder (VLAE), introduced in 2016, addresses this by assigning different kinds of information to different parts of the model: its latent code carries global structure, while an image decoder such as PixelCNN supplies local detail. section 7.1.1 section 7.1.1
The VLAE makes that division of labor work through two design choices:
- It limits the decoder’s receptive field so it focuses on local details rather than reconstructing the whole image without using the code. section 7.1.3
- It uses an autoregressive prior: latent components are modeled one at a time, allowing later components to depend on earlier ones instead of assuming independence. section 7.1.1
A lossy code for images
For MNIST, the code preserves essentials such as an image’s shape and arrangement; the decoder fills in local detail. Comparing original images with VLAE reconstructions makes this trade-off visible. section 7.1.2
Table 7.1 Same small-receptive-field PixelCNN with different priors
|
Model
|
NLL (nats)
|
|---|---|
|
PixelCNN decoder
|
87.55
|
|
PixelCNN decoder + Gaussian prior
|
79.48
|
|
PixelCNN decoder + autoregressive flow prior
|
78.94
|
Compare the same small-receptive-field PixelCNN decoder with no prior, a Gaussian prior, and an autoregressive-flow prior. The reported negative log-likelihood falls from 87.55 to 79.48 to 78.94 nats. section 7.1.3
The VLAE performed strongly on MNIST, Omniglot, and Caltech-101 Silhouettes, and remained competitive on CIFAR-10. Its broader lesson is that architecture can shape what information a model retains and where that information lives. section 7.1.2 section 7.1.3
