A latent image is a lower-dimensional tensor used inside an image-generation model, rather than a final image represented in pixel space. In Stable Diffusion, generated latents have shape 4 × 64 × 64; a VAE decoder converts them into 3 × 512 × 512 images. section 10.4 section 10.5
In text-to-image generation, the latent image changes in stages:
- Start with random noise in latent space. section 9.1.2
- Iteratively denoise it with a U-Net, conditioned on text embeddings; the scheduler controls the denoising steps. section 10.3 section 10.4
- Decode the final latent with a VAE to produce the high-resolution image. section 10.5
Figure 9.8 Generated latent images in time steps 800, 600, . . ., and 0. These images are generated with a pretrained LDM. We start with an image with pure random noise at t = 1,000. We use the diffusion model to denoise it iteratively and obtain the noisy image at t = 800, which is shown in the far left of the figure. We continue to denoise the image and obtain images at t = 600, 400, and 200. Finally, we obtain the image at t = 0, which is shown at the far right of the figure. The images have a shape of 4 × 64 × 64.
Compare latent images at successive denoising steps, from noisy intermediates to a clean latent. section 9.3.3
A latent takes shape
For a prompt describing a banana riding a motorcycle, intermediate latents gradually reveal the prompted scene before the VAE decodes the final image. section 9.3.3 section 9.1.2
