9 Autoencoders

 

This chapter covers

  • Understanding the design principles and patterns for DNN and CNN autoencoders
  • Coding these models using the procedural design pattern
  • Regularization when training an autoencoder
  • Using an autoencoder for compression, denoising, and super resolution
  • Using an autoencoder for pretraining to improve the model’s ability to generalize

Up to now, we’ve discussed only models for supervised learning. An autoencoder model falls into the category of unsupervised learning. As a reminder, in supervised learning our data consists of the features (for example, image data) and labels (for example, classes), and we train the model to learn to predict the labels from the features. In unsupervised learning, we either have no labels or don’t use them, and we train the model to find correlating patterns in the data. You might ask, what can we do without labels? We can do a lot of things, and autoencoders are one type of model architecture that can learn from unlabeled data.

Autoencoders are the fundamental deep learning models for unsupervised learning. Even without human labeling, autoencoders can learn image compression, representational learning, image denoising, super-resolution and pretext tasks—and we’ll cover each of these in this chapter.

9.1 Deep neural network autoencoders

9.1.1 Autoencoder architecture

9.1.2 Encoder

9.1.3 Decoder

9.1.4 Training

9.2 Convolutional autoencoders

9.2.1 Architecture

9.2.2 Encoder

9.2.3 Decoder

9.3 Sparse autoencoders

9.4 Denoising autoencoders

9.5 Super-resolution

9.5.1 Pre-upsampling SR

9.5.2 Post-upsampling SR