3 Convolutional and residual neural networks

 

This chapter covers

  • Understanding the structure of convolutional neural networks
  • Constructing a ConvNet model
  • Designing and constructing a VGG model
  • Designing and constructing a residual network model

Chapter 2 introduced the fundamentals behind deep neural networks (DNNs), a network architecture based on dense layers. We also demonstrated how to make a simple image classifier using dense layers, and discussed the limitations when attempting to scale a DNN to larger sizes of images. The introduction of neural networks using convolutional layers for feature extraction and learning, known as convolutional neural networks (CNNs), made it possible to scale image classifiers for practical applications.

This chapter covers the design patterns, and evolution in the design patterns, for early SOTA convolutional neural networks. We cover three design patterns in this chapter, in sequence of their evolution:

  • ConvNet
  • VGG
  • Residual networks

3.1 Convolutional neural networks

3.1.1 Why we use a CNN over a DNN for image models

3.1.2 Downsampling (resizing)

3.1.3 Feature detection

3.1.4 Pooling

3.1.5 Flattening

3.2 The ConvNet design for a CNN

3.3 VGG networks

3.4 ResNet networks

3.4.1 Architecture

3.4.2 Batch normalization

Summary