concept VAE in category deep learning

This is an excerpt from Manning's book Deep Learning with JavaScript: Neural networks in TensorFlow.js. Login to get full access to this book.

The previous section gave you a quick tour of how deep learning can be used to generate sequential data such as text. In the remaining parts of this chapter, we will look at how to build neural networks to generate images. We will examine two types of models: variational autoencoder (VAE) and generative adversarial network (GAN). Compared to a GAN, the VAE has a longer history and is structurally simpler. So, it forms a good on-ramp for you to get into the fast-moving world of deep-learning-based image generation.

This example consists of two parts: training the VAE in Node.js and using the VAE decoder to generate images in the browser. To start the training part, use

The VAE algorithm randomly samples a latent vector from the latent normal distribution by using a vector called epsilon—a random vector of the same length as zMean and zLogVar. In simple math equations, this step, which is referred to as reparameterization in the literature, looks like

z = zMean + exp(zLogVar * 0.5) * epsilon

copy

The multiplication by 0.5 converts the variance to the standard deviation, which is based on the fact that the standard deviation is the square root of the variance. The equivalent JavaScript code is

z = zMean.add(zLogVar.mul(0.5).exp().mul(epsilon));

copy

(See listing 10.3.) Then, z will be fed to the decoder portion of the VAE so that an output image can be generated.

As listing 10.4 shows, ZLayer is instantiated and gets used as a part of the encoder. The encoder is written as a functional model, instead of the simpler sequential model, because it has a nonlinear internal structure and produces three outputs: zMean, zLogVar, and z (see the schematic in figure 10.7). The encoder outputs z because it will get used by the decoder, but why does the encoder include zMean and zLogVar in the outputs? It’s because they will be used to calculate the loss function of the VAE, as you will see shortly.

Figure 10.7. Schematic illustration of the TensorFlow.js implementation of VAE, including the internal details of the encoder and decoder parts and the custom loss function and optimizer that support VAE training.