An image encoder turns an image into a latent representation; decoding takes that representation back into image space. In a classical autoencoder, the decoder produces an output with the same dimensions as the input, and training uses the input image as the reconstruction target. The practical question is whether the latent representation retains enough information to make that reconstruction useful. section 17.1.2
A variational autoencoder (VAE) changes what reaches the decoder. Its encoder produces a mean and variance rather than one fixed code. A sampling step draws a latent point from the resulting distribution, and the decoder turns that point into an image. This arrangement supports a continuous, structured latent space that can be sampled for image generation. section 17.1.2 section 17.1.3
Decode an MNIST digit
A Keras VAE uses a two-dimensional latent point as decoder input. A Dense layer expands it to 7 × 7 × 64 values, Reshape makes those values image-shaped, and two Conv2DTranspose layers enlarge the feature maps. A final Conv2D layer with sigmoid activation produces an image matching the 28 × 28 × 1 input shape. After training, passing points from a two-dimensional grid to vae.decoder.predict produces a grid of digit images. This makes the latent space visible: nearby sampled points can be compared through their decoded outputs. section 17.1.3
Listing 17.3 VAE decoder network, mapping latent space points to images
latent_inputs = keras.Input(shape=(latent_dim,)) #1 x = layers.Dense(7 * 7 * 64, activation="relu")(latent_inputs) #2 x = layers.Reshape((7, 7, 64))(x) #3 x = layers.Conv2DTranspose(64, 3, activation="relu", strides=2, padding="same")( x #4 ) #4 x = layers.Conv2DTranspose(32, 3, activation="relu", strides=2, padding="same")( #4 x #4 ) #4 decoder_outputs = layers.Conv2D(1, 3, activation="sigmoid", padding="same")(x) #5 decoder = keras.Model(latent_inputs, decoder_outputs, name="decoder")
The decoder listing shows the concrete path from a latent vector through Dense, Reshape, and convolutional layers to an image output. section 17.1.3
Rule of thumb: distinguish reconstructing an encoded image from generating an image at a chosen latent point. The same decoder handles both; the source of its latent input changes. “Decoder” also appears in sequence-to-sequence models, but there it predicts target tokens from earlier target tokens and an encoded source sequence. A decoder-only language model removes that separate encoder and generates from a single sequence instead. section 17.1.2 section 17.1.3 section 15.2 section 16.2.1
