A latent space is where a model represents input information in a transformed, often lower-dimensional form. In an autoencoder, an encoder compresses an input into a latent code, and a decoder uses that code to reconstruct it. The bottleneck encourages the code to preserve features needed for reconstruction rather than every detail. section 7.1
That compression can make the representation useful beyond reconstruction: a code for a handwritten digit may retain edges, loops, and stroke direction that support later classification. But a plain autoencoder can organize valid codes into disconnected clusters. Interpolating or sampling through the gaps may then produce meaningless outputs, a problem called off-manifold sampling. section 7.1 section 7.1
Variational autoencoders aim for a smoother latent space, but a powerful decoder may ignore the code, a failure known as posterior collapse. The variational lossy autoencoder (VLAE), introduced by Peter (Xi) Chen, Ilya Sutskever, and colleagues in 2016, assigns global structure—such as object identity and arrangement—to the code, while a PixelCNN-style decoder supplies local detail. section 7.1 section 7.1.1
Figure 7.2 Left: plain autoencoders yield disjoint clusters in the latent space, leaving significant gaps. Interpolations must traverse off-manifold regions. Right: VAEs produce continuous, overlapping regions that enable smooth interpolation and reliable sampling.
Compare the gaps in a plain autoencoder’s latent space with the continuous, overlapping regions of a VAE. section 7.1
