Variational autoencoder

As explained in Build a Text-to-Image Generator (from Scratch)

A variational autoencoder is a generative neural model that encodes data into a lower-dimensional probabilistic latent representation and decodes that representation into reconstructed or generated data.

A variational autoencoder (VAE) is a generative neural model with an encoder and a decoder. Unlike a traditional autoencoder, which maps an image to a fixed latent vector, its encoder describes a probability distribution in a compact latent space. Sampling from that distribution lets the decoder produce reconstructions with variation. section 11.2.2

Follow an image through the model:

• Encode: turn the image into parameters of a latent distribution, typically a normal distribution described by a mean and standard deviation. section 11.2.2
• Sample and decode: draw a latent representation, then reconstruct an image from it. A smooth latent space supports transitions between representations. section 9.1.1
• Train: combine reconstruction loss, which measures image differences, with KL divergence, which measures departure from a prior distribution. Both losses guide parameter updates. section 11.2.2

Compress and check the reconstruction

Center-crop and resize a beaver image and a dolphin image to 3 × 512 × 512. A pretrained encoder samples a 4 × 64 × 64 latent for each; the decoder returns 3 × 512 × 512 reconstructions. Compare each reconstruction with its original rather than treating the output shape alone as proof that detail was preserved. This workflow uses pretrained weights, not training a VAE from scratch. section 9.2.1 section 9.2.2 section 11.2

N=torch.distributions.Normal(0, 1)
noise=N.sample(torch.Size([1,4,64,64]))    #1
latent1=encoder(image1.unsqueeze(0),
    noise)[0]    
latent2=encoder(image2.unsqueeze(0),
    noise)[0]    #2
print(latent1.shape,latent2.shape)    #3

The encoder call pairs an image tensor with sampled normal noise and produces a compact latent tensor. section 9.2.2

A latent diffusion model uses this compression to do costly denoising in a smaller space, then passes the generated latent to a VAE decoder for the final high-resolution image. For example, reducing 3 × 512 × 512 to 4 × 64 × 64 makes the diffusion input 48 times smaller. The trade-off is potential lost detail: conventional VAEs can also produce blurry reconstructions, a limitation addressed differently by vector-quantized VAEs. section 9.0 section 11.6

Figure 11.4 The architecture of a variational autoencoder (VAE) and the training steps involved in generating high-resolution color images via the VAE’s two primary components: an encoder (upper-middle left) and a decoder (lower-middle right)

Trace the encoder, latent sampling, decoder and training losses together before using the components separately. section 11.2.2

How Build a Text-to-Image Generator (from Scratch) covers it

How deep it goes

Explains probabilistic encoding and reconstruction-plus-KL training, then uses pretrained components; does not build or train a VAE from scratch.

Examples

Compares beaver and dolphin originals, latents and reconstructions, then decodes text-guided latent images.

What it assumes

Introduces autoencoders before VAEs, while the image exercises use Python, PyTorch tensors and pretrained weights.

What you can do after

Apply it via pretrained PyTorch components and diffusers AutoencoderKL to compress, reconstruct and decode image latents.

How it connects

Connects autoencoders and KL divergence to latent diffusion, with vector-quantized VAEs as an alternative.

liveBookmanning.com

Read Build a Text-to-Image Generator (from Scratch) on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Causal AIA variational autoencoder is a generative neural-network model that combines an encoder-defined approximating distribution with a decoder-defined generative distribution over observed data and latent variables.
• 

  liveBookmanning.com
  Learn Generative AI with PyTorchA generative model that learns an encoder-decoder latent-variable representation so it can reconstruct inputs and sample new outputs from latent space.
• 

  liveBookmanning.com
  Outlier Detection in PythonA variational autoencoder is a generative neural-network model that learns a probabilistic latent representation and reconstructs or generates observations from it.
• 

  liveBookmanning.com
  Math and Architectures of Deep LearningAn autoencoder that models the latent space probabilistically and is trained with regularizing constraints.
• 

  liveBookmanning.com
  Deep Learning with Python, Third EditionA generative latent-variable model that learns an encoder-decoder structure and a probabilistic latent space for reconstructing and sampling data.
• 

  liveBookmanning.com
  Evolutionary Deep Learning: Genetic algorithms and neural networksA variational autoencoder is a neural network that encodes inputs as probability distributions, samples from those distributions, and decodes the samples to reconstruct or generate data.
• 

  liveBookmanning.com
  Generative AI in ActionA variational autoencoder is a generative neural model that encodes data into a probabilistic latent representation and decodes samples from that representation into reconstructed or newly generated data.
• 

  liveBookmanning.com
  Deep Learning with R, Third EditionA variational autoencoder is a probabilistic autoencoder whose encoder represents inputs as parameters of a latent probability distribution and whose decoder reconstructs or generates data from sampled latent points.
• 

  liveBookmanning.com
  Machine Learning Algorithms in DepthA variational autoencoder is a generative neural network that learns an approximate latent-variable posterior and reconstructs observations from sampled latent representations.
• 

  liveBookmanning.com
  Machine Learning for Tabular Data: XGBoost, Deep Learning, and AIA variational autoencoder is a neural network architecture that encodes data into a probabilistic latent space and decodes latent representations to reconstruct or generate data.
• 

  liveBookmanning.com
  Data Without LabelsA variational autoencoder is a neural-network generative model that learns a probabilistic latent representation and reconstructs data from sampled latent variables.
• 

  liveBookmanning.com
  Graph Neural Networks in ActionA variational autoencoder is a probabilistic autoencoder that learns a parameterized latent-variable distribution and decodes samples from it to reconstruct or generate data.

Compare how 13 books explain variational autoencoder →

Related concepts

• encoderhas part · An encoder is a component that transforms input data into an encoded representation for subsequent processing, reconstruction, comparison, or transmission.
• autoencoderkind of · An autoencoder is a neural network that encodes input data into an internal representation and decodes it to reconstruct the input or an approximation.
• decodehas part · Decoding is the process of transforming an encoded or internal representation into an interpretable or target-form output.
• generationkind of · Generation is the computational production of outputs from supplied inputs or context, often by applying a model to produce predictions or new content.
• reconstruction lossuses · Reconstruction loss is a measure of the discrepancy between an input or observation and the output a model reconstructs from it or its representation.
• image generationuses · Image generation is the computational creation of new images from learned models, input data, or specified conditions.

This explainer was generated by Manning's knowledge graph from the text of Build a Text-to-Image Generator (from Scratch). The section links point to the parts of the book it draws on.