Autoencoder

As explained in Learn Generative AI with PyTorch

A neural network that learns to compress inputs into a latent representation and reconstruct them, used here for digit-image generation and reconstruction.

An autoencoder (AE) is a neural network that learns to compress an input and reconstruct it. Its encoder maps the input to a smaller representation in latent space; its decoder uses that representation to produce a reconstruction. Training reduces the difference between the original and reconstructed inputs. Because each input supplies its own reconstruction target, an AE can learn representations from unlabeled data. section 7.1.1 section 7.0

A standard AE maps an image to a deterministic point in latent space. A variational autoencoder instead encodes a probability distribution. The simpler AE makes the compression-and-reconstruction loop a useful starting point before working with variational autoencoders. section 7.1.2 section 7.0

For handwritten digits, build and train the loop in small steps:

• Load MNIST’s 60,000 training images and 10,000 test images. Convert pixel values from 0–255 to tensors in the range 0–1, then make batches of 32. section 7.2.1
• Flatten each 28 × 28 grayscale image into 784 values. Dense layers compress it through 200 hidden values to a 20-value latent vector; decoder layers expand that vector back to 784 output values. section 7.2.2
• Compare output pixels with input pixels using squared reconstruction error, backpropagate the loss, and update the model with Adam. The example trains for 10 epochs rather than tuning against a validation set. section 7.2.2

Check what the model learned

Keep one test image for each digit, 0 through 9, and plot the originals above their reconstructions after each epoch. Before training, the reconstructions look like noise. This side-by-side check makes improvement visible without treating reconstruction loss as the only measure. section 7.2.2

Figure 7.3 The architecture of an AE and the steps to train one to generate handwritten digits. An AE consists of an encoder (middle left) and a decoder (middle right). In each iteration of training, images of handwritten digits are fed to the encoder (step 1). The encoder compresses the images to deterministic points in the latent space (step 2). The decoder takes the encoded vectors (step 3) from the latent space and reconstructs the images (step 4). The AE adjusts its parameters to minimize the reconstruction loss, the difference between the originals and the reconstructions (step 5).

Follow one image through encoding, its deterministic latent vector, decoding, and the reconstruction-loss update. section 7.1.2

Compression can discard information, leaving imperfect reconstructions. Dense layers suit this small grayscale-digit example; higher-resolution color images usually call for convolutional layers in the encoder and decoder. section 7.1.1 section 7.1.2

How Learn Generative AI with PyTorch covers it

How deep it goes

Works through dense-layer architecture, data preparation, squared-error training, and visual checks; defers validation-based parameter tuning.

Examples

Builds a dense-layer MNIST autoencoder and plots ten original digits beside their reconstructions.

What it assumes

Assumes basic PyTorch and neural-network training familiarity while defining the encoder, decoder, and latent space before implementation.

What you can do after

Build it from scratch in PyTorch to encode handwritten digits, reconstruct them, and visually compare outputs with originals.

How it connects

Connects latent space and reconstruction error to dimensionality reduction, denoising, variational autoencoders, and encoder–decoder Transformers.

liveBookmanning.com

Read Learn Generative AI with PyTorch on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Inside Deep Learning: Math, Algorithms, ModelsA neural network trained to reconstruct its input, typically by compressing it into a lower-dimensional latent representation and decoding it back.
• 

  liveBookmanning.com
  Outlier Detection in PythonAn autoencoder is a neural network trained to encode an input into a representation and reconstruct the original input.
• 

  liveBookmanning.com
  Math and Architectures of Deep LearningA neural network model trained to compress inputs into a latent representation and reconstruct the original inputs from that representation.
• 

  liveBookmanning.com
  Data Without LabelsAn autoencoder is a neural-network model that learns to encode data into a lower-dimensional representation and reconstruct the original input from it.
• 

  liveBookmanning.com
  Evolutionary Deep Learning: Genetic algorithms and neural networksAn autoencoder is a neural network that learns to encode inputs into a representation and decode that representation to reconstruct the inputs.
• 

  liveBookmanning.com
  Graph Neural Networks in ActionAn autoencoder is a neural network that encodes inputs into latent representations and decodes those representations to reconstruct the inputs.
• 

  liveBookmanning.com
  Delivering Machine Learning Projects: From design to deploymentAn autoencoder is a neural network that learns to encode inputs into a lower-dimensional representation and reconstruct the original inputs.
• 

  liveBookmanning.com
  Financial AI in PracticeAn autoencoder is a neural network trained to reconstruct its input, typically by encoding it into a lower-dimensional representation and decoding it.
• 

  liveBookmanning.com
  Deep Learning with R, Third EditionAn autoencoder is a neural-network model that learns to encode inputs into representations and decode those representations back into approximations of the inputs.
• 

  liveBookmanning.com
  Deep Learning with Python, Third EditionA neural network trained to reconstruct its input, often by compressing and then decoding it, used here for denoising images.
• 

  liveBookmanning.com
  Introduction to Generative AI, Second EditionAn autoencoder is a neural network that learns to encode data into a compressed representation and reconstruct the original data from it.

Compare how 12 books explain autoencoder →

Related concepts

• encoderhas part · An encoder is a component that transforms input data into an encoded representation for subsequent processing, reconstruction, comparison, or transmission.
• decoderhas part · A decoder is a component that maps encoded data or representations to decoded values, reconstructed data, or generated output.
• variational autoencoderincludes · A variational autoencoder is a generative neural network that learns a probabilistic latent representation of data and decodes samples from it to reconstruct or generate data.
• deep learningkind of · Deep learning is a machine-learning approach that uses neural networks with multiple computational layers to learn representations from data.
• neural networkkind of · A neural network is a computational model of interconnected, parameterized units that learns transformations from inputs to outputs.
• Kerasuses · Keras is an open-source, high-level deep-learning API for building, training, evaluating, and deploying neural-network models.

This explainer was generated by Manning's knowledge graph from the text of Learn Generative AI with PyTorch. The section links point to the parts of the book it draws on.