Wasserstein distance

As explained in Evolutionary Deep Learning: Genetic algorithms and neural networks

A distance between probability distributions that measures the minimum transport cost required to transform one distribution into another.

Wasserstein distance measures how much transport is required to transform one probability distribution into another. In generative adversarial networks, it provides an earthmover-style loss that compares real and generated data through a scaled distance rather than probability uncertainty. section 9.3.1

The GAN formulation changes the training signal in several practical ways:

• Replaces the discriminator’s real/fake probability with a critic score representing how real or fake an image is. section 9.3.1
• Measures vertical as well as horizontal differences between distributions, unlike the probabilistic comparison described for Kullback–Leibler and Jensen–Shannon distances. section 9.3.1
• Aims to make GAN training more robust and less sensitive while reducing mode collapse, convergence failure, and vanishing gradients. section 9.3 section 9.3.1

Using Wasserstein loss in a DCGAN

A DCGAN implementation renames the discriminator as a critic, compiles it with wasserstein_loss, and computes the loss as the mean of true labels multiplied by predictions. Both generator and critic use RMSprop with a learning rate of 0.00005. After each critic iteration, clipping every layer’s weights to ±clip_value limits exploding gradients. The experiment trains on one extracted MNIST digit class for 80 epochs and can be switched to Fashion-MNIST. section 9.3.2

Figure 9.10 The difference between variational distance and Wasserstein distance

Compare probabilistic distance with earthmover distance through the supplied distribution diagram. section 9.3.1

How Evolutionary Deep Learning: Genetic algorithms and neural networks covers it

How deep it goes

Explains the critic concept and distribution intuition, then implements loss, optimizer, and weight clipping without formal derivation.

Examples

Demonstrates a Wasserstein DCGAN with MNIST digits, optional Fashion-MNIST data, RMSprop, critic training, and clipped weights.

What it assumes

Assumes familiarity with GANs and DCGAN training while presenting the Wasserstein-loss changes incrementally through an existing implementation.

What you can do after

Apply it via Python and Keras by adapting a DCGAN with a critic, Wasserstein loss, shared RMSprop settings, and weight clipping.

How it connects

Connects Wasserstein distance to GANs, critics, generators, discriminators, mode collapse, vanishing gradients, and convergence failure.

liveBookmanning.com

Read Evolutionary Deep Learning: Genetic algorithms and neural networks on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Learn Generative AI with PyTorchA distance measure used in Wasserstein GAN training to compare distributions and provide a smoother optimization signal than standard GAN losses.
• 

  liveBookmanning.com
  Machine Learning Platform EngineeringWasserstein distance is a measure of dissimilarity between probability distributions based on the minimum cost of transforming one distribution into another.
• 

  liveBookmanning.com
  Machine Learning System DesignWasserstein distance is a metric between probability distributions defined by the minimum cost of transporting mass from one distribution to another.

Compare how 4 books explain Wasserstein distance →

Related concepts

• Wasserstein GANused by · A Wasserstein GAN is a generative adversarial network that uses a critic and a Wasserstein-based objective to compare generated and real-data distributions.
• WGAN-GPused by · WGAN-GP is a Wasserstein generative adversarial network variant that adds a gradient penalty to constrain the critic and stabilize training.
• population stability indexalternative to · The population stability index (PSI) is a measure of the difference between two probability distributions, often used to quantify shifts in a population or model inputs.
• probability distributionmeasures · A probability distribution is a mathematical specification assigning probabilities to the possible values or outcomes of a random variable, with total probability one.
• driftmeasures · Drift is a change over time in a system’s behavior, state, data, or configuration relative to a reference or expected baseline.

This explainer was generated by Manning's knowledge graph from the text of Evolutionary Deep Learning: Genetic algorithms and neural networks. The section links point to the parts of the book it draws on.