A variational autoencoder (VAE) learns a structured latent space that can be sampled to generate new data. Like an autoencoder, it encodes an input and decodes a representation back into a reconstruction. Unlike a classical autoencoder, it encodes an input as parameters of a probability distribution, then samples a latent point to decode. section 17.1.2 section 17.4
Generate MNIST digits
A Keras VAE can encode 28-by-28 MNIST images into a two-dimensional latent space. After training, decode a grid of latent points to see how the generated digits change across that space. section 17.1.3
The model has three main parts:
- The encoder uses strided convolutions to map an image to two vectors: a mean and a log variance. Strides downsample while retaining information about where features appear. section 17.1.3
- The sampler draws random noise and combines it with the mean and log variance to produce a latent point. section 17.1.3
- The decoder reshapes the latent point and uses convolution layers to produce an image with the original dimensions. section 17.1.3
Listing 17.2 Latent space sampling layer
from keras import ops class Sampler(keras.Layer): def __init__(self, **kwargs): super().__init__(**kwargs) self.seed_generator = keras.random.SeedGenerator() #1 self.built = True def call(self, z_mean, z_log_var): batch_size = ops.shape(z_mean)[0] z_size = ops.shape(z_mean)[1] epsilon = keras.random.normal( (batch_size, z_size), seed=self.seed_generator #2 ) #2 return z_mean + ops.exp(0.5 * z_log_var) * epsilon #3
The Keras sampling layer shows how random noise is scaled by the encoded log variance and added to the mean. section 17.1.3
Training is self-supervised: the input images are also the reconstruction targets. The Keras implementation uses a custom compute_loss() method and tracks reconstruction and KL losses. Once trained, the decoder can generate images from chosen latent points without first encoding an input image. section 17.1.3 section 17.4
Listing 17.6 Sampling a grid of points from the 2D latent space and decoding them to images
import matplotlib.pyplot as plt n = 30 #1 digit_size = 28 figure = np.zeros((digit_size * n, digit_size * n)) grid_x = np.linspace(-1, 1, n) #2 grid_y = np.linspace(-1, 1, n)[::-1] #2 for i, yi in enumerate(grid_y): #3 for j, xi in enumerate(grid_x): #3 z_sample = np.array([[xi, yi]]) #4 x_decoded = vae.decoder.predict(z_sample) #4 digit = x_decoded[0].reshape(digit_size, digit_size) #4 figure[ i * digit_size : (i + 1) * digit_size, j * digit_size : (j + 1) * digit_size, ] = digit plt.figure(figsize=(15, 15)) start_range = digit_size // 2 end_range = n * digit_size + start_range pixel_range = np.arange(start_range, end_range, digit_size) sample_range_x = np.round(grid_x, 1) sample_range_y = np.round(grid_y, 1) plt.xticks(pixel_range, sample_range_x) plt.yticks(pixel_range, sample_range_y) plt.xlabel("z[0]") plt.ylabel("z[1]") plt.axis("off") plt.imshow(figure, cmap="Greys_r")
This grid-sampling example decodes points across the two-dimensional latent space into MNIST images. section 17.1.3
VAEs are useful when reconstruction, interpretable latent spaces, or image editing matters. Their continuous latent spaces can support effects such as changing a frown into a smile or smoothly morphing between images. For high-fidelity image generation, diffusion models are generally the stronger choice. section 17.1.2 section 17.4












