A discriminator is the detection half of a generative adversarial network (GAN). It takes a sample and predicts whether it came from the training data (real) or the generator (fake). The generator has the opposite job: make samples the discriminator will mistake for real ones. Their repeated competition gives the generator feedback for improving its output. section 1.2.1 section 3.3
Start with two-value samples
Suppose real training samples are pairs (x, y) on an exponential-growth curve. A PyTorch discriminator can take two inputs, pass them through dense layers of 256, 128, and 64 units with ReLU and dropout, then produce one sigmoid output. That output is p, the predicted probability that the pair is real; 1 − p is the predicted probability that it is fake. The first layer must match the sample’s two values, while the final layer supplies the single real-or-fake prediction. section 3.0 section 3.3.1
Listing 3.3 Creating a discriminator network
import torch.nn as nn
device="cuda" if torch.cuda.is_available() else "cpu" #1
D=nn.Sequential(
nn.Linear(2,256), #2
nn.ReLU(),
nn.Dropout(0.3), #3
nn.Linear(256,128),
nn.ReLU(),
nn.Dropout(0.3),
nn.Linear(128,64),
nn.ReLU(),
nn.Dropout(0.3),
nn.Linear(64,1), #4
nn.Sigmoid()).to(device)
The PyTorch listing shows the two-input dense network, dropout layers, and final sigmoid in one runnable architecture. section 3.3.1
Train the two networks in alternating steps:
- Label real samples 1 and generated samples 0. For each batch, compare the discriminator’s predictions with the appropriate labels, backpropagate the loss, and update its parameters. Clear accumulated gradients before each update. section 3.4.1
- Then generate another fake batch and pass it through the discriminator, but compare its predictions with real labels when updating the generator. The generator is rewarded for producing samples judged real. section 3.4.1
- Inspect generated samples as training proceeds. In the curve example, plotting fake points against real points shows whether the emerging shape matches the training data. section 3.4.1
The discriminator’s input changes with the task. A multiples-of-five example uses 100-value one-hot inputs and a one-output classifier. For 28 × 28 grayscale clothing images, it takes 784 flattened pixel values. High-resolution color anime faces instead use a convolutional discriminator: its filters extract spatial features from a three-channel image, with batch normalization and LeakyReLU in later layers. The image generator can be designed by mirroring the discriminator’s layers in reverse. section 3.5.2 section 4.1.2 section 4.5.1
Listing 4.4 A discriminator in DCGAN
import torch.nn as nn
import torch
device = "cuda" if torch.cuda.is_available() else "cpu"
D = nn.Sequential(
nn.Conv2d(3, 64, 4, 2, 1, bias=False), #1
nn.LeakyReLU(0.2, inplace=True), #2
nn.Conv2d(64, 128, 4, 2, 1, bias=False),
nn.BatchNorm2d(128), #3
nn.LeakyReLU(0.2, inplace=True),
nn.Conv2d(128, 256, 4, 2, 1, bias=False),
nn.BatchNorm2d(256),
nn.LeakyReLU(0.2, inplace=True),
nn.Conv2d(256, 512, 4, 2, 1, bias=False),
nn.BatchNorm2d(512),
nn.LeakyReLU(0.2, inplace=True),
nn.Conv2d(512, 1, 4, 1, 0, bias=False),
nn.Sigmoid(),
nn.Flatten()).to(device) #4
The DCGAN listing makes the shift from dense inputs to convolutional color-image classification concrete. section 4.5.1
A Wasserstein-style conditional GAN uses a related network called a critic instead of this real-probability classifier. Given an image and its condition, the critic aims to rate real images highly and fake images poorly; its loss also includes a gradient penalty. In either setup, the judging network is needed for training, not for producing new samples afterward: keep the trained generator and discard the discriminator or critic. section 5.3.3 section 3.6 section 5.4.2
