Interpolation generates representations between two latent points, then decodes them into outputs that transition smoothly from one input to another. In a variational autoencoder (VAE), the points are encodings produced by the encoder and interpreted by the decoder. section 7.3.2 section 7.4.4
A practical interpolation workflow uses a trained VAE:
- Choose two encodings in the latent space, such as average encodings for women with and without glasses. section 7.4.4
- Combine them with a weight: z = w·encoding₂ + (1−w)·encoding₁. section 7.4.4
Transitioning between faces
Set w to 0, 0.2, 0.4, 0.6, 0.8, and 1.0 for the encodings of women with and without glasses. Decoding these six mixtures produces a sequence that gradually changes from a woman with glasses to a woman without glasses. section 7.4.4
Figure 7.6 The architecture of a VAE and the steps to train one to generate human face images. A VAE consists of an encoder (middle upper left) and a decoder (middle bottom right). In each iteration of training, human face images are fed to the encoder (step 1). The encoder compresses the images to probabilistic points in the latent space (step 2; since we assume normal distributions, each probability point is characterized by a vector of means and a vector of standard deviations). We then sample encodings from the distribution and present them to the decoder. The decoder takes sampled encodings (step 3) and reconstructs images (step 4). The VAE adjusts its parameters to minimize the sum of reconstruction loss and the KL divergence. The KL divergence measures the difference between the encoder’s output and a standard normal distribution.
Use the architecture diagram to follow the encoder–latent-space–decoder path that makes interpolated encodings meaningful. section 7.3.2
