A latent space is a compact, lower-dimensional representation of encoded data. A variational autoencoder (VAE) encodes an image into numerical latents, then decodes those latents back into an image. Stable Diffusion does most of its work on latents, producing ordinary pixels only at the beginning and end. section 4.3.1
Latent representations make image generation practical by:
- Shrinking the working representation dramatically: a 512×512 image has 262,144 pixels, while the example reduces it to 64 latent “pixels,” each actually a number. section 4.3.1
- Balancing compactness with useful generalization rather than merely preserving every detail like JPEG compression. section 4.3.1
- Allowing model changes to affect related concepts: a LoRA trained on Keatsie can alter the latent region associated with dogs and make other dogs resemble Keatsie. section 13.5.4
From Metropolis image to latents
Encode a 512×512 frame from Metropolis into a compact numerical representation, then decode it. The reconstruction can look close to the original while losing or misplacing details such as the eyes—a reminder that imperfect memory can be useful for generation. section 4.3.1
Figure 4.14 The VAE transforms the image into latents

Visualizes how a VAE transforms an ordinary image into latents. section 4.3.1





















