Adversarial training

As explained in Build a Text-to-Image Generator (from Scratch)

Adversarial training is a training procedure in which competing models are optimized against each other, typically with a generator producing samples and a discriminator judging them.

Adversarial training optimizes competing models against each other: a generator creates samples, while a discriminator judges whether images are real or generated. In VQGAN, a VQ-VAE supplies the generator, combining discrete latent codes with adversarial refinement for perceptually realistic reconstructions. section 11.4.2 section 11.6

A VQGAN training process links three mechanisms:

• The encoder maps an image to continuous representations, the codebook replaces them with nearest discrete vectors, and the decoder reconstructs the image. section 11.4.2
• The discriminator evaluates real and generated images, encouraging decoded outputs to appear perceptually convincing rather than merely pixelwise similar. section 11.4.2
• The objective combines reconstruction, adversarial, and quantization losses; LPIPS can measure perceptual reconstruction quality. section 11.4.2

VQGAN image generation

An image is encoded into discrete latent tokens, decoded into a reconstruction, and refined through discriminator feedback. This preserves structured compression while improving visual fidelity and supports later image generation from symbolic sequences. section 11.4.2 section 11.6

Figure 11.6 VQGAN architecture and training process in which VQGAN combines a VQ-VAE generator with a discriminator in a GAN framework

Trace the VQ-VAE generator, codebook, decoder, and discriminator through the adversarial training loop. section 11.4.2

How Build a Text-to-Image Generator (from Scratch) covers it

How deep it goes

Explains the generator, discriminator, codebook, and blended losses, but does not provide implementation code or optimization schedules.

Examples

Uses VQGAN image reconstruction, discrete image tokens, and DALL-E-style symbolic image generation as concrete applications.

What it assumes

Assumes familiarity with VQ-VAEs, GAN generators and discriminators, latent representations, and image reconstruction.

What you can do after

Evaluate VQGAN designs by tracing adversarial feedback alongside reconstruction and quantization objectives.

How it connects

Connects GANs with VQ-VAEs, discrete latent representations, codebooks, LPIPS, and transformer-based image generation.

liveBookmanning.com

Read Build a Text-to-Image Generator (from Scratch) on liveBook Get the book at manning.com

Related concepts

• generative adversarial networkused by · A generative adversarial network is a generative model with a generator and discriminator trained in competition, enabling the generator to produce samples resembling a target dataset or distribution.
• trainingkind of · Training is a process of developing or improving a person’s or model’s capabilities through instruction, experience, or adjustment using data.
• VQGANused by · VQGAN is a generative model that combines vector-quantized discrete latent representations with adversarial and perceptual objectives to synthesize images.
• hallucinationaddresses · A hallucination is model-generated content that is false, fabricated, or unsupported by relevant evidence but presented as plausible or factual.
• adversarial inputcontrasts with · An input crafted, selected, or modified to exploit a system’s weaknesses or induce incorrect, unsafe, or otherwise unintended behavior.

This explainer was generated by Manning's knowledge graph from the text of Build a Text-to-Image Generator (from Scratch). The section links point to the parts of the book it draws on.