CycleGAN translates images between two visual domains without needing matched before-and-after examples. Instead of generating an image from random noise, it takes an image from one domain as input: a face with black hair can become a face with blond hair, and the reverse translation works too. section 6.0 section 6.5
The model has two generators and two discriminators. One generator maps domain A to B; the other maps B to A. Each discriminator judges whether images in its domain look real. Training combines these adversarial losses with cycle consistency loss: translate an image to the other domain and back, then compare the reconstruction with the original. This encourages the translation to retain key features rather than merely produce a plausible image in the target domain. section 6.1 section 6.1.1
Figure 6.1 The architecture of a CycleGAN to convert images with black hair to ones with blond hair and to convert images with blond hair to ones with black hair. The diagram also outlines the training steps to minimize adversarial losses. How the model minimizes cycle consistency losses is explained in figure 6.2.
The architecture diagram traces both hair-color translations and shows where each domain’s discriminator supplies an adversarial training signal. section 6.1.1
Translate hair color without paired portraits
Separate celebrity portraits into black-hair and blond-hair sets; no portrait needs a matching version with the other color. The example uses 48,472 black-hair images and 29,980 blond-hair images. During training, the blond-hair generator transforms a black-hair image, and the blond-hair discriminator compares generated and real blond-hair images. Training also runs in the opposite direction. A round trip—black to blond to black—lets you compare a reconstruction directly with its starting image. section 6.1.1 section 6.2.1 section 6.4.2
For an implementation, label the domains A and B so the same model can be trained on different image datasets. Construct two generators and two discriminators, optimize the generators and discriminators, and use the trained generators for translation; the discriminators are discarded afterward. The generator implementation uses convolutional downsampling, residual blocks, and upsampling. An eyeglasses dataset provides another task for the same two-domain setup. section 6.3 section 6.3.2 section 6.4
Listing 6.8 Round-trip conversions of images with black or blond hair
gen_A.load_state_dict(torch.load("files/gen_black.pth",
map_location=device))
gen_B.load_state_dict(torch.load("files/gen_blond.pth",
map_location=device))
i=1
for black,blond in loader:
fake_blond=gen_B(black.to(device))
save_image(black*0.5+0.5,f"files/black{i}.png") #1
save_image(fake_blond*0.5+0.5,f"files/fakeblond{i}.png")
fake2black=gen_A(fake_blond)
save_image(fake2black*0.5+0.5,
f"files/fake2black{i}.png") #2
fake_black=gen_A(blond.to(device))
save_image(blond*0.5+0.5,f"files/blond{i}.png") #3
save_image(fake_black*0.5+0.5,f"files/fakeblack{i}.png")
fake2blond=gen_B(fake_black)
save_image(fake2blond*0.5+0.5,
f"files/fake2blond{i}.png") #4
i=i+1
if i>10:
break
The conversion listing loads both trained generators and saves originals, translations, and round-trip reconstructions in each direction. section 6.4.2
