A latent space holds the vectors a generative model turns into outputs. A vector matters through the model that interprets it: change the vector and a trained generator may produce a different face, shape, or piece of music. In an autoencoder, the route begins with an input: an encoder compresses it into a latent representation, and a decoder uses that representation to reconstruct it. section 3.1 section 7.0
The route into latent space depends on the model:
- A GAN draws random noise vectors and gives them directly to its generator. For a simple (x, y) output, the input can be two-dimensional; an image generator instead takes a 100-value vector and produces 784 values that reshape into a 28 × 28 image. section 3.3.2 section 4.1.2
- An autoencoder derives its vector from an existing input. A handwritten-digit encoder reduces 784 pixel values to a 20-value latent variable; its decoder reconstructs the digit. Training compares reconstructed pixels with the originals. section 7.2.2
- A variational autoencoder (VAE) encodes a face as a probability distribution described by means and standard deviations. It samples a vector for the decoder, training with both reconstruction loss and KL divergence. After training, randomly drawn vectors can also yield new faces. section 7.3.2 section 7.4.3
Change a face by interpolating vectors
Save two noise vectors that produce faces with different features. For weights 0, 0.25, 0.5, 0.75, and 1, feed each weighted average of the vectors to the trained generator. In the worked face example, male features gradually give way to female features. A conditional GAN offers a separate control: attach an eyeglasses label to the vector. Unlike selecting one handpicked vector, a label can specify that characteristic across many generated faces, but training the conditional model requires labeled data. section 5.5.2 section 5.5
Listing 5.11 Vector arithmetic to select image characteristics
weights=[0,0.25,0.5,0.75,1] #1
plt.figure(figsize=(20,4),dpi=50)
for i in range(5):
ax = plt.subplot(1, 5, i + 1)
# change the value of z
z=weights[i]*z_female_ng+(1-weights[i])*z_male_ng #2
noise_and_labels=torch.cat(
[z.reshape(1, z_dim, 1, 1),
labels_ng[0].reshape(1, 2, 1, 1)],dim=1).to(device)
fake=generator(noise_and_labels).cpu().detach() #3
img=(fake[0]/2+0.5).permute(1,2,0)
plt.imshow(img)
plt.xticks([])
plt.yticks([])
plt.subplots_adjust(wspace=-0.08,hspace=-0.01)
plt.show()
The listing builds five weighted-average noise vectors, joins each with a fixed no-eyeglasses label, and passes them to the generator. Keeping the label fixed isolates the effect of changing the latent vector. section 5.5.2
Latent inputs need not be a single vector. MuseGAN draws four sets of noise vectors—chords, style, melody, and groove—to control different parts of a multitrack, two-bar music piece. The practical question is always what a particular trained generator or decoder does with its latent inputs, not what an isolated coordinate means. section 13.2 section 3.1
