Adversarial loss is a training objective that rewards generated images for looking convincingly real. In VQGAN, a discriminator evaluates decoded images as real or fake while the generator learns to fool it. section 11.4.2
VQGAN combines adversarial loss with reconstruction or perceptual loss and quantization loss. These objectives divide the work: adversarial loss promotes perceptual realism, perceptual loss preserves visual similarity, and quantization loss keeps encoder outputs close to codebook vectors. section 11.6 section 11.4.2
Trace the adversarial mechanism through the VQGAN pipeline:
- The VQ-VAE generator encodes an image, quantizes its latent representation into discrete codes, and decodes those codes into a reconstruction. section 11.0
- The discriminator compares real training images with generated reconstructions and learns to distinguish real samples from fake ones. section 11.4.2
- The generator receives adversarial pressure to produce decoded images that can fool the discriminator and appear perceptually convincing. section 11.4.2
VQGAN reconstruction
A VQGAN decodes discrete codebook vectors into an image. Reconstruction and perceptual objectives keep it visually similar to the input, while adversarial loss pushes the result toward the appearance of a realistic training sample. section 11.4.2
Figure 11.6 VQGAN architecture and training process in which VQGAN combines a VQ-VAE generator with a discriminator in a GAN framework
Use the architecture figure to follow how the VQ-VAE generator and discriminator interact during adversarial training. section 11.4.2
