VQ-VAE generator

As explained in Build a Text-to-Image Generator (from Scratch)

A VQ-VAE generator is the VQGAN component consisting of an encoder, a codebook, and a decoder that reconstructs images from discrete latent codes.

A VQ-VAE generator is the VQGAN component that converts images into discrete latent codes and reconstructs them. It combines an encoder, a learned codebook, and a decoder. section 11.4.2

Trace the reconstruction pipeline:

• The encoder maps an input image x to a continuous latent representation z. section 11.4.2
• The codebook replaces each encoder output with its nearest learned embedding, producing discrete codes zq. section 11.4.2
• The decoder transforms zq into a reconstructed image x̂. section 11.4.2

Within VQGAN, adversarial training refines these reconstructions through a discriminator. The combined objective uses reconstruction, adversarial, and quantization losses; LPIPS measures perceptual reconstruction similarity. section 11.4.2

Figure 11.6 VQGAN architecture and training process in which VQGAN combines a VQ-VAE generator with a discriminator in a GAN framework

Use the architecture figure to follow the encoder–codebook–decoder path and its interaction with the discriminator. section 11.4.2

How Build a Text-to-Image Generator (from Scratch) covers it

How deep it goes

Explains the encoder, codebook, decoder, quantization path, and three-part training objective without implementation details.

Examples

Uses input image x, latent representations z and zq, reconstructed image x̂, and a codebook example of 1,024 embeddings.

What you can do after

Evaluate how discrete codes, perceptual losses, and adversarial training shape VQGAN image reconstruction.

How it connects

Connects encoder, codebook, decoder, discriminator, GAN training, quantization, and LPIPS.

liveBookmanning.com

Read Build a Text-to-Image Generator (from Scratch) on liveBook Get the book at manning.com

Related concepts

• VQGANpart of · VQGAN is a generative model that combines vector-quantized discrete latent representations with adversarial and perceptual objectives to synthesize images.

This explainer was generated by Manning's knowledge graph from the text of Build a Text-to-Image Generator (from Scratch). The section links point to the parts of the book it draws on.