A generative adversarial network (GAN) creates new data that resembles its training data without simply copying examples. It fits one-to-many problems, such as producing several valid images for the same concept, and can learn useful representations without labeled data. section 9.0
A GAN contains two competing subnetworks. The generator G maps a sampled latent vector z to synthetic data, while the discriminator D classifies inputs as real or generated. Latent dimensions are chosen as a hyperparameter; Gaussian samples such as zᵢ ∼ N(0,1) provide an easy source of new generations. section 9.1
Train the networks in alternating steps:
- Give D a batch of real data labeled 1 and generated data labeled 0, compute both classification losses, and update D. section 9.1.2 section 9.1.3
- Sample z, compute G(z), and train G through D using the real label so that generated samples become harder to distinguish from real data. section 9.1.2 section 9.1.3
- Repeat the game: improving G forces D to improve, while improving D supplies a stronger signal for G. section 9.1.2
A first fully connected GAN
A PyTorch implementation can use a 128-dimensional latent vector, 512-neuron hidden layers, and an output reshaped to 28 × 28 for MNIST. Define G and D as modules, use BCEWithLogitsLoss for real-versus-fake classification, and give each network its own AdamW optimizer. Apply a sigmoid to G when generated values must match MNIST's [0,1] range. During training, detach G(z) for D's update, then reuse the attached fake output for G's update. Plot generated digits, discriminator scores, and both losses to inspect progress. section 9.1.3
GAN training can suffer from mode collapse, where G concentrates on common or easy patterns instead of producing diverse outputs. Wasserstein GANs address this with a discriminator penalty and remove the discriminator's sigmoid output to reduce vanishing-gradient problems. Convolutional architectures can improve image results when their structure matches the problem. section 9.3 section 9.4 section 9.9
The latent representation can support controlled generation. Interpolating between two latent vectors produces intermediate outputs, while discovered semantic vectors can alter recurring properties such as smiles, hair color, or hairstyle. Conditional models add labels when explicit control is needed. section 9.6 section 9.6.2 section 9.0
Figure 9.3 Diagram of how GANs are generally trained. The generator G learns to assign meaning to random values z to produce meaningful outputs 
. The discriminator receives real and fake data but doesn’t know which—and tries to learn the difference in a standard classification problem. Then G’s loss is computed on D’s output but is the inverse. G gets an error when D does not, and G gets no error when D does.
Trace how random latent values become generated outputs, how D compares real and fake data, and how the opposing losses update G and D. section 9.1 section 9.1.2
