chapter three

3 Wide Convolutional Neural Networks

 

This chapter covers

  • The wide convolutional layer design pattern.
  • Why researchers went wide vs deep.
  • Refactoring micro architecture patterns to decrease computational complexity.
  • Coding former state-of-the-art (SOTA) wide convolutional models with the procedural design pattern

Up to now in the book, we’ve focused on networks with deeper layers, block layers and shortcuts in residual networks for image related tasks (classification, object localization, image segmentation). Starting in 2014 with Inception v1 (GoogLeNet) (https://arxiv.org/abs/1409.4842) and 2015 with ResNeXt (Microsoft Research) (https://arxiv.org/abs/1611.05431) and Inception v2, neural network designs moved into wide layers, reducing the need for going deeper in layers.  Essentially, a wide layer is having multiple convolutions in parallel and then concatenating their outputs; whereas deeper layers have sequential convolutions and aggregate their outputs.

3.1      Inception V1 (GoogLeNet)

3.1.1                     Naive Inception Module

3.1.2                     Inception v1 Module

3.1.3                     Stem

3.1.4                     Learner

3.1.5                     Auxiliary Classifiers

3.1.6                     Classifier

3.2      Inception V2 - Factoring Convolutions

3.3      Inception V3 - Architecture Redesign

3.3.1                     Inception Groups & Blocks

3.3.2                     Normal Convolution

3.3.3                     Spatial Separable Convolution

3.3.4                     Stem

3.3.5                     Auxiliary Classifier

3.4      ResNeXt - Wide Residual Neural Networks

3.4.1                     ResNeXt Block

3.4.2                     Architecture

3.5      Wide Residual Network

3.5.1                     Architecture

3.5.2                     Wide Residual Block

3.6      Summary