liveBook · Manning

Reconstruction loss

As explained in Evolutionary Deep Learning: Genetic algorithms and neural networks

Reconstruction loss is a loss value measuring the discrepancy between an input example and the output reconstructed from it.

Reconstruction loss measures how closely a model’s output reproduces its input. In an autoencoder, pixel-wise mean squared error (MSE) provides a simple comparison; a variational autoencoder (VAE) combines this term with Kullback–Leibler divergence to represent distributional learning. section 8.5.2

Common implementation choices include:

  • Use MSE for pixel-wise reconstruction comparison. section 8.5.2
  • Use binary cross entropy as an alternative reconstruction measure, then compare training behavior after normalizing loss values. section 8.5.2
  • Add reconstruction loss to KL loss to form the VAE objective. section 8.5.2

Keras VAE loss

Flatten the input and output tensors, calculate binary cross entropy or MSE, scale the reconstruction term by image size, add KL divergence, and register the mean with add_loss.

How Evolutionary Deep Learning: Genetic algorithms and neural networks covers it

How deep it goes
Defines MSE and binary cross entropy implementation choices, combines them with KL divergence, and defers statistical-distance theory.
Examples
Demonstrates MSE and binary cross entropy inside a Keras VAE loss, with normalized metrics for comparing their scales.
What it assumes
Assumes encoder-decoder models and introduces reconstruction comparison before adding distributional KL divergence.
What you can do after
Implement reconstruction and combined VAE losses in Keras, then compare MSE and binary cross entropy during training.
How it connects
Connects autoencoders, variational autoencoders, mean squared error, binary cross entropy, and Kullback-Leibler divergence.
Cover of Evolutionary Deep Learning: Genetic algorithms and neural networks
liveBookmanning.com
Read Evolutionary Deep Learning: Genetic algorithms and neural networks on liveBook Get the book at manning.com

Also explained in

Compare how 10 books explain reconstruction loss →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Evolutionary Deep Learning: Genetic algorithms and neural networks. The section links point to the parts of the book it draws on.