liveBook · Manning

Latent space

As explained in Local Image Generation

A lower-dimensional representation space in which encoded data is represented by numerical latent variables.

A latent space is a compact, lower-dimensional representation of encoded data. A variational autoencoder (VAE) encodes an image into numerical latents, then decodes those latents back into an image. Stable Diffusion does most of its work on latents, producing ordinary pixels only at the beginning and end. section 4.3.1

Latent representations make image generation practical by:

  • Shrinking the working representation dramatically: a 512×512 image has 262,144 pixels, while the example reduces it to 64 latent “pixels,” each actually a number. section 4.3.1
  • Balancing compactness with useful generalization rather than merely preserving every detail like JPEG compression. section 4.3.1
  • Allowing model changes to affect related concepts: a LoRA trained on Keatsie can alter the latent region associated with dogs and make other dogs resemble Keatsie. section 13.5.4

From Metropolis image to latents

Encode a 512×512 frame from Metropolis into a compact numerical representation, then decode it. The reconstruction can look close to the original while losing or misplacing details such as the eyes—a reminder that imperfect memory can be useful for generation. section 4.3.1

Figure 4.14 The VAE transforms the image into latents

Visualizes how a VAE transforms an ordinary image into latents. section 4.3.1

How Local Image Generation covers it

How deep it goes
Explains encoding, decoding, compression, scale reduction, and reconstruction limits without detailing VAE internals.
Examples
Uses a Metropolis frame, 512×512-to-64 latent reduction, imperfect reconstruction, and Keatsie-related dog changes.
What it assumes
Introduces images, pixels, encoding, decoding, and latents from basic examples without assuming data-science expertise.
What you can do after
Evaluate how image models trade pixel-level fidelity for compact representations and why LoRA changes can spill into related concepts.
How it connects
Connects latent space with variational autoencoders, encoding, decoding, Stable Diffusion, diffusion models, and LoRA.

Also explained in

Compare how 22 books explain latent space →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Local Image Generation. The section links point to the parts of the book it draws on.