How deeply does Inside Deep Learning: Math, Algorithms, Models cover WGAN-GP?

Implements the training loop, interpolation, autograd calculation, penalty, and optimizer updates without deriving the Wasserstein objective.

WGAN-GP

As explained in Inside Deep Learning: Math, Algorithms, Models

A Wasserstein GAN variant that adds a gradient penalty to stabilize discriminator training.

WGAN-GP stabilizes GAN training by adding a gradient penalty to the discriminator loss. It reduces opportunities for vanishing gradients and helps the generator keep up with the discriminator, addressing mode collapse. section 9.3.3

A training step follows this sequence:

Compute discriminator scores for real data and generated data without detaching the generated samples. section 9.3.3

Toy grid and handwritten digits

The same training function is applied first to a two-dimensional toy grid and then to 28×28 digit data, with generated samples, discriminator scores, and smoothed losses inspected afterward.

How Inside Deep Learning: Math, Algorithms, Models covers it

Cover of Inside Deep Learning: Math, Algorithms, Models

liveBook manning.com

Read Inside Deep Learning: Math, Algorithms, Models on liveBook Get the book at manning.com

Related concepts

gradient penaltyuses · A gradient penalty is a regularization loss term that penalizes undesirable gradients during model optimization.
Wasserstein distanceuses · Wasserstein distance is a distance between probability distributions defined by the minimum cost of transporting probability mass between them under a specified ground distance.
generative adversarial networkalternative to · A generative adversarial network is a generative model with a generator and discriminator trained in competition, enabling the generator to produce samples resembling a target dataset or distribution.
mode collapseaddressed by · Mode collapse is a generative-model failure in which outputs concentrate on a narrow range of patterns and fail to represent the diversity of the target distribution.
discriminatorcontrasts with · A discriminator is the model component of a generative adversarial network that distinguishes real data from data generated by its generator.
conditional GANalternative to · A generative adversarial network whose generation and discrimination are conditioned on auxiliary information, such as class labels, to guide the produced outputs.

This explainer was generated by Manning's knowledge graph from the text of Inside Deep Learning: Math, Algorithms, Models. The section links point to the parts of the book it draws on.