A generator is the network in a generative adversarial network (GAN) that turns a latent input, usually random noise, into a synthetic sample. Its goal is to make samples that resemble the training data. The discriminator has the opposite job: distinguish generated samples from real ones. Changing the latent input can produce different outputs. section 1.2.1 section 1.2.2
Start with pairs of numbers
To generate points along an exponential growth curve, feed a two-value noise vector into a small PyTorch network: Linear(2, 16), ReLU, Linear(16, 32), ReLU, Linear(32, 2). The two outputs are an (x, y) pair. During training, the discriminator judges whether such pairs resemble real points on the curve. section 3.3.2 section 3.3
Train the generator by passing its fake samples through the discriminator and comparing the discriminator’s predictions with labels for real samples. Backpropagate that loss and update the generator. The real labels matter: the generator is trying to make the discriminator call its samples real. Compare generated and training samples as training progresses; early points may show no meaningful shape before they align with the curve. section 3.4.1
Listing 3.8 Defining the train_G() function
def train_G():
noise=torch.randn((batch_size,2))
noise=noise.to(device)
optimG.zero_grad()
fake_samples=G(noise) #1
out_G=D(fake_samples) #2
loss_G=loss_fn(out_G,real_labels) #3
loss_G.backward()
optimG.step() #4
return loss_G, fake_samples
The training function shows the full generator update: sample noise, generate fake data, obtain discriminator predictions, compute loss against real labels, and step the generator optimizer. section 3.4.1
The output format determines the generator design:
- For 28 × 28 grayscale clothing images, dense layers expand 100 noise values into 784 output values, which can be reshaped into an image. section 4.1.2
- For color anime faces, a DCGAN uses transposed convolutions to build higher-resolution images; its final layer outputs three color channels. section 4.5.1
- For patterned integers, a trained generator can be saved, reloaded, and used without the discriminator to produce multiples of five. section 3.5.2 section 3.5.4
- For unpaired image translation, CycleGAN uses two generators, one for each direction between image domains. MuseGAN instead combines noise inputs for chords, style, melody, and groove to produce music. section 6.5 section 13.2.2
Listing 4.5 Designing a generator in DCGAN
G=nn.Sequential(
nn.ConvTranspose2d(100, 512, 4, 1, 0, bias=False), #1
nn.BatchNorm2d(512),
nn.ReLU(inplace=True),
nn.ConvTranspose2d(512, 256, 4, 2, 1, bias=False), #2
nn.BatchNorm2d(256),
nn.ReLU(inplace=True),
nn.ConvTranspose2d(256, 128, 4, 2, 1, bias=False),
nn.BatchNorm2d(128),
nn.ReLU(inplace=True),
nn.ConvTranspose2d(128, 64, 4, 2, 1, bias=False),
nn.BatchNorm2d(64),
nn.ReLU(inplace=True),
nn.ConvTranspose2d(64, 3, 4, 2, 1, bias=False), #3
nn.Tanh()).to(device) #4
The DCGAN listing makes the image-generation design concrete: transposed-convolution layers expand a 100-channel input to a three-channel output, followed by Tanh. section 4.5.1
Once training is complete, keep the generator to create new samples; the discriminator is no longer needed for generation. What it produces depends on the training data, whether those data are number patterns, images, or music. section 3.6 section 13.2.2
