chapter six

6 Wide Convolutional Neural Networks

 

This chapter covers

  • The wide convolutional layer design pattern
  • Understanding the advantages of wide vs deep layers
  • Refactoring micro architecture patterns to decrease computational complexity
  • Coding former SOTA wide convolutional models with the procedural design pattern

Up to now, we’ve focused on networks with deeper layers, block layers and shortcuts in residual networks for image-related tasks such as classification, object localization, and image segmentation). Now we are going to take a look at networks with wide, rather than deep, convolutional layers. Starting in 2014 with Inception v1 (GoogLeNet) and 2015 with ResNeXt (Microsoft Research) and Inception v2, neural network designs moved into wide layers, reducing the need for going deeper in layers. Essentially, a wide layer design means having multiple convolutions in parallel and then concatenating their outputs.In contrast, deeper layers have sequential convolutions and aggregate their outputs.

6.1    Inception V1 (GoogLeNet)

6.1.1   Naive Inception Module

6.1.2   Inception v1 Module

6.1.3   Stem

6.1.4   Learner

6.1.5   Auxiliary Classifiers

6.1.6   Classifier

6.2    Inception V2 - Factoring Convolutions

6.3    Inception V3 - Architecture Redesign

6.3.1   Inception Groups and Blocks

6.3.2   Normal Convolution

6.3.3   Spatial Separable Convolution

6.3.4   Stem Redesign and Implementation

6.3.5   Auxiliary Classifier

6.4    ResNeXt - Wide Residual Neural Networks

6.4.1   ResNeXt Block

6.4.2   ResNeXt Architecture

6.5    Wide Residual Network

6.5.1   WRN-50-2 Architecture

6.5.2   Wide Residual Block

6.6    Summary