LPIPS is a perceptual image-similarity metric that compares learned features from pretrained neural networks rather than raw pixel differences, aligning more closely with human judgments of visual quality. section 11.4.2
LPIPS fits into image reconstruction in two stages:
- A pretrained network such as VGG or AlexNet extracts features from real and generated images. section 11.4.2
- Feature differences provide a perceptual reconstruction loss that complements adversarial and quantization losses. section 11.4.2
VQGAN reconstruction
In VQGAN, LPIPS helps ensure that decoded images resemble their originals perceptually, while adversarial training encourages realistic appearance and quantization keeps encoder outputs near codebook vectors. section 11.4.2
Figure 11.6 VQGAN architecture and training process in which VQGAN combines a VQ-VAE generator with a discriminator in a GAN framework
Trace LPIPS's role as the perceptual reconstruction component within the VQ-VAE generator and discriminator training process. section 11.4.2
