Perceptual loss measures visual similarity between generated and real images by comparing learned representations rather than only corresponding pixel values. It aims to reflect how similar images look to human observers, making it useful when pixel-wise agreement does not fully capture visual quality. section 11.4.2
LPIPS is a common perceptual loss. It extracts features with a pretrained deep network, such as VGG or AlexNet, and compares those features between two images. This representation-based comparison captures perceptual realism more effectively than raw-pixel differences alone. section 11.4.2
Within VQGAN, perceptual loss contributes to reconstruction quality alongside adversarial and quantization losses. The adversarial term encourages convincing images, while the quantization term keeps encoder outputs close to codebook vectors; perceptual loss preserves visual fidelity. section 11.4.2 section 11.6
VQGAN reconstruction
A VQGAN reconstructs an image after encoding it, assigning latent patches to codebook entries, and decoding the resulting discrete codes. LPIPS evaluates the reconstruction against the original through deep features, helping favor an image that looks right rather than one that merely matches pixels. section 11.4.2 section 11.0
Figure 11.6 VQGAN architecture and training process in which VQGAN combines a VQ-VAE generator with a discriminator in a GAN framework
Trace how perceptual loss fits into VQGAN training alongside the generator, discriminator, reconstruction, and quantization components. section 11.4.2
