A variational autoencoder (VAE) is the part of an image-generation pipeline that turns ordinary images into smaller latent representations, then turns latents back into images. Stable Diffusion does most of its work on these latents rather than full-size images, reducing the number of calculations required. section 4.3.1 section 4.5
The VAE has two jobs: encoding shrinks an image into latent space, and decoding reconstructs an image from those latents. Reconstruction need not preserve every original detail; losing some information helps the system learn more general visual patterns instead of memorizing a single image. section 4.3.1
A VAE differs from ordinary image compression in ways that matter for generation:
- It learns a compression method for a specific task, balancing smaller representations with useful generalization. section 4.3.1
- It makes denoising cheaper because the diffusion process can operate on compressed latents. section 4.3.1 section 4.5
- It can be replaced to change image appearance; a different VAE can support anime-like results. section 4.4
From image to latent and back
An input image is encoded into latents, processed during generation, and decoded into the final image. The reconstructed result can be close to the original while showing small differences, such as altered eyes—a useful reminder that the VAE is not a perfect photocopier. section 4.3.1
Figure 4.11 Closer look at the variational autoencoder.

Inspect the VAE’s position between ordinary images and latent representations in the generation pipeline. section 4.3.1
In a configurable workflow, the VAE is a separate component rather than an invisible detail. A Flux setup, for example, installs `ae.safetensors` in the ComfyUI VAE directory alongside separate text-encoder and U-NET files. section 10.1.2














