Autoencoder

As explained in Evolutionary Deep Learning: Genetic algorithms and neural networks

An autoencoder is a neural network that learns to encode inputs into a representation and decode that representation to reconstruct the inputs.

An autoencoder (AE) is an unsupervised neural network that learns to reproduce its input. An encoder funnels data through a narrow latent representation; a decoder rebuilds the original form from that encoding. Training compares input and output, commonly with mean squared or pixel-wise loss, so the bottleneck learns features that capture the input. section 8.1.1 section 8.6

Figure 8.1 A convolutional AE

The convolutional AE layout makes the funnel explicit: convolution and pooling reduce an image to a latent view, while upsampling restores its spatial size. section 8.1.1 section 8.1.2

For 28×28 single-channel images, a practical convolutional AE stacks Conv2D and MaxPool layers in the encoder, ending in the latent view. The decoder applies convolution followed by UpSampling repeatedly, then uses a final one-channel convolutional output. Combine the input and output in a Keras model and compile it with Adam and MSE. section 8.1.2 section 8.1.2

Image reconstruction and latent-space inspection

Train on unlabeled Fashion-MNIST images, then inspect reconstructed outputs and project learned encodings into two dimensions with t-SNE. A digit example shows latent-space clustering by classes even though labels are not supplied during AE training. section 8.1.1 section 8.1.2

Useful experiments keep the encoder and decoder structurally balanced while changing:

• The dataset, including MNIST handwritten digits. section 8.1.3
• Learning rate and batch size to observe training effects. section 8.1.3
• The number of convolutional layers on both sides of the model. section 8.1.3
• Generalization layers such as dropout and batch normalization. section 8.1.3

Architecture can also be evolved with a genetic algorithm. Encode encoder convolution layers, decoder up-convolution layers, batch normalization, and dropout as gene tokens. Generate a decoder with the same number of main layers as the encoder, then use crossover and mutation while preserving the AE funnel. Evaluate each individual by building and training its model, using validation loss as fitness to minimize. This is computationally slow, so reduced datasets can demonstrate the workflow; full data is preferable for optimization. section 8.2.1 section 8.3 section 8.4

How Evolutionary Deep Learning: Genetic algorithms and neural networks covers it

How deep it goes

Works through a convolutional Keras build, training, genetic encoding, mutation, crossover, and validation-loss evaluation; variational AEs are only summarized.

Examples

Uses Fashion-MNIST training images, MNIST latent-space visualization with t-SNE, and evolved convolutional architectures.

What it assumes

Assumes prior CNN and evolutionary-algorithm code familiarity while introducing AE reconstruction and latent representations directly.

What you can do after

Build it from scratch as a convolutional Keras autoencoder and evolve balanced encoder-decoder architectures with DEAP.

How it connects

Connects autoencoders with unsupervised learning, representation learning, convolutional neural networks, variational autoencoders, and genetic algorithms.

liveBookmanning.com

Read Evolutionary Deep Learning: Genetic algorithms and neural networks on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Inside Deep Learning: Math, Algorithms, ModelsA neural network trained to reconstruct its input, typically by compressing it into a lower-dimensional latent representation and decoding it back.
• 

  liveBookmanning.com
  Outlier Detection in PythonAn autoencoder is a neural network trained to encode an input into a representation and reconstruct the original input.
• 

  liveBookmanning.com
  Math and Architectures of Deep LearningA neural network model trained to compress inputs into a latent representation and reconstruct the original inputs from that representation.
• 

  liveBookmanning.com
  Data Without LabelsAn autoencoder is a neural-network model that learns to encode data into a lower-dimensional representation and reconstruct the original input from it.
• 

  liveBookmanning.com
  Learn Generative AI with PyTorchA neural network that learns to compress inputs into a latent representation and reconstruct them, used here for digit-image generation and reconstruction.
• 

  liveBookmanning.com
  Graph Neural Networks in ActionAn autoencoder is a neural network that encodes inputs into latent representations and decodes those representations to reconstruct the inputs.
• 

  liveBookmanning.com
  Delivering Machine Learning Projects: From design to deploymentAn autoencoder is a neural network that learns to encode inputs into a lower-dimensional representation and reconstruct the original inputs.
• 

  liveBookmanning.com
  Financial AI in PracticeAn autoencoder is a neural network trained to reconstruct its input, typically by encoding it into a lower-dimensional representation and decoding it.
• 

  liveBookmanning.com
  Deep Learning with R, Third EditionAn autoencoder is a neural-network model that learns to encode inputs into representations and decode those representations back into approximations of the inputs.
• 

  liveBookmanning.com
  Deep Learning with Python, Third EditionA neural network trained to reconstruct its input, often by compressing and then decoding it, used here for denoising images.
• 

  liveBookmanning.com
  Introduction to Generative AI, Second EditionAn autoencoder is a neural network that learns to encode data into a compressed representation and reconstruct the original data from it.

Compare how 12 books explain autoencoder →

Related concepts

• encoderhas part · An encoder is a component that transforms input data into an encoded representation for subsequent processing, reconstruction, comparison, or transmission.
• variational autoencoderincludes · A variational autoencoder is a generative neural network that learns a probabilistic latent representation of data and decodes samples from it to reconstruct or generate data.
• decoderhas part · A decoder is a component that maps encoded data or representations to decoded values, reconstructed data, or generated output.
• deep learningkind of · Deep learning is a machine-learning approach that uses neural networks with multiple computational layers to learn representations from data.
• neural networkkind of · A neural network is a computational model of interconnected, parameterized units that learns transformations from inputs to outputs.
• Kerasuses · Keras is an open-source, high-level deep-learning API for building, training, evaluating, and deploying neural-network models.

This explainer was generated by Manning's knowledge graph from the text of Evolutionary Deep Learning: Genetic algorithms and neural networks. The section links point to the parts of the book it draws on.