6 Wide convolutional neural networks

 

This chapter covers

  • Introducing the wide convolutional layer design pattern
  • Understanding the advantages of wide versus deep layers
  • Refactoring micro-architecture patterns to decrease computational complexity
  • Coding former SOTA wide convolutional models with the procedural design pattern

Up to now, we’ve focused on networks with deeper layers, block layers, and shortcuts in residual networks for image-related tasks such as classification, object localization, and image segmentation. Now we are going to take a look at networks with wide, rather than deep, convolutional layers. Starting in 2014 with Inception v1 (GoogLeNet) and 2015 with ResNeXt and Inception v2, neural network designs moved into wide layers, reducing the need for going deeper in layers. Essentially, a wide-layer design means having multiple convolutions in parallel and then concatenating their outputs. In contrast, deeper layers have sequential con-volutions and aggregate their outputs.

So what led to the experimentation with wide-layer design patterns? At the time, researchers understood that for models to gain in accuracy, they needed more capacity. More specifically, they needed to have overcapacity for redundancy.

6.1 Inception v1

6.1.1 Naive inception module

6.1.2 Inception v1 module

6.1.3 Stem

6.1.4 Learner

6.1.5 Auxiliary classifiers

6.1.6 Classifier

6.2 Inception v2: Factoring convolutions

6.3 Inception v3: Architecture redesign

6.3.1 Inception groups and blocks

6.3.2 Normal convolution

6.3.3 Spatial separable convolution

6.3.4 Stem redesign and implementation

6.3.5 Auxiliary classifier