Autoencoder

As explained in Data Without Labels

An autoencoder is a neural-network model that learns to encode data into a lower-dimensional representation and reconstruct the original input from it.

An autoencoder is a neural network that learns a compact encoding of its input and then tries to reconstruct that input. It is useful when you want features from unlabeled data, such as images, without designing every feature by hand. Common uses include dimensionality reduction, anomaly detection, and denoising. section 9.2 section 9.3 section 9.6

Data passes through three components:

• The encoder reduces the input to a smaller representation. section 9.4
• The bottleneck, or code, limits how much information can pass through. section 9.4
• The decoder uses that code to reconstruct an output with the same dimensions as the input. section 9.4 section 9.5

Figure 9.1 Structure of an autoencoder with an encoder, a bottleneck, and a decoder

The architecture diagram shows where compression occurs and how the decoder turns the code back into a reconstruction. section 9.4

Train the network by comparing each reconstruction with its input and reducing the reconstruction loss. Choose the code size, network depth, nodes per layer, and loss function. A smaller code forces greater compression; additional depth increases complexity and processing time. Binary cross-entropy is suggested for inputs in the [0,1] range, and mean squared error otherwise. Compression is harder when input variables have little relationship to one another. section 9.5

Reconstructing handwritten digits

With Keras, load MNIST digits, scale pixel values by dividing by 255, and flatten each image to 784 values. Map those values through a 32-value encoded layer and a 784-value decoded layer. Fit with each training image serving as both input and target, then encode and decode test images. Plot original and reconstructed digits side by side to inspect what the compact code preserved. section 9.8

Reconstruction error—the difference between the input and its reconstruction—can also help identify anomalies. An undercomplete autoencoder uses a compressed bottleneck for dimensionality reduction. Unlike linear PCA, an autoencoder with nonlinear activations can learn nonlinear relationships; with only linear layers, the undercomplete version is equivalent to PCA. Other named variants include sparse, contractive, denoising, and variational autoencoders. section 9.6 section 9.7

Check the match between training data and intended use before deploying one: reconstruction loses information, and an autoencoder trained on one kind of image may not work on another. For general-purpose data compression, established approaches such as JPEG can be difficult to beat. section 9.4 section 9.6

How Data Without Labels covers it

How deep it goes

Explains encoder–bottleneck–decoder training choices and implements a basic model, while leaving the mathematics of variants to further reading.

Examples

Builds a Keras autoencoder for MNIST digits and plots test images beside their reconstructions.

What it assumes

Assumes some familiarity with neural networks and PCA, but defines the autoencoder components and points unfamiliar readers to hyperparameter guidance.

What you can do after

Apply it via Keras to train a basic digit autoencoder and inspect reconstructions on test data.

How it connects

Connects autoencoders to feature learning, principal component analysis, reconstruction loss, and anomaly detection.

liveBookmanning.com

Read Data Without Labels on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Inside Deep Learning: Math, Algorithms, ModelsA neural network trained to reconstruct its input, typically by compressing it into a lower-dimensional latent representation and decoding it back.
• 

  liveBookmanning.com
  Outlier Detection in PythonAn autoencoder is a neural network trained to encode an input into a representation and reconstruct the original input.
• 

  liveBookmanning.com
  Math and Architectures of Deep LearningA neural network model trained to compress inputs into a latent representation and reconstruct the original inputs from that representation.
• 

  liveBookmanning.com
  Evolutionary Deep Learning: Genetic algorithms and neural networksAn autoencoder is a neural network that learns to encode inputs into a representation and decode that representation to reconstruct the inputs.
• 

  liveBookmanning.com
  Learn Generative AI with PyTorchA neural network that learns to compress inputs into a latent representation and reconstruct them, used here for digit-image generation and reconstruction.
• 

  liveBookmanning.com
  Graph Neural Networks in ActionAn autoencoder is a neural network that encodes inputs into latent representations and decodes those representations to reconstruct the inputs.
• 

  liveBookmanning.com
  Delivering Machine Learning Projects: From design to deploymentAn autoencoder is a neural network that learns to encode inputs into a lower-dimensional representation and reconstruct the original inputs.
• 

  liveBookmanning.com
  Financial AI in PracticeAn autoencoder is a neural network trained to reconstruct its input, typically by encoding it into a lower-dimensional representation and decoding it.
• 

  liveBookmanning.com
  Deep Learning with R, Third EditionAn autoencoder is a neural-network model that learns to encode inputs into representations and decode those representations back into approximations of the inputs.
• 

  liveBookmanning.com
  Deep Learning with Python, Third EditionA neural network trained to reconstruct its input, often by compressing and then decoding it, used here for denoising images.
• 

  liveBookmanning.com
  Introduction to Generative AI, Second EditionAn autoencoder is a neural network that learns to encode data into a compressed representation and reconstruct the original data from it.

Compare how 12 books explain autoencoder →

Related concepts

• encoderhas part · An encoder is a component that transforms input data into an encoded representation for subsequent processing, reconstruction, comparison, or transmission.
• decoderhas part · A decoder is a component that maps encoded data or representations to decoded values, reconstructed data, or generated output.
• variational autoencoderincludes · A variational autoencoder is a generative neural network that learns a probabilistic latent representation of data and decodes samples from it to reconstruct or generate data.
• deep learningkind of · Deep learning is a machine-learning approach that uses neural networks with multiple computational layers to learn representations from data.
• neural networkkind of · A neural network is a computational model of interconnected, parameterized units that learns transformations from inputs to outputs.
• Kerasuses · Keras is an open-source, high-level deep-learning API for building, training, evaluating, and deploying neural-network models.

This explainer was generated by Manning's knowledge graph from the text of Data Without Labels. The section links point to the parts of the book it draws on.