Network-in-network paradigm

As explained in Math and Architectures of Deep Learning

A design pattern that replaces a single large convolution with multiple smaller and parallel transformations inside a network block.

The network-in-network paradigm replaces a single fixed-scale convolutional choice with parallel transformations using different kernel sizes. This lets an Inception module respond to objects appearing at different spatial scales while increasing depth and width within a fixed computational budget. section 11.2.2

The design addresses two architectural trade-offs:

• Avoids relying on one kernel size when image content may appear close up or far away. section 11.2.2
• Combines outputs from parallel branches, commonly using 1×1, 3×3, and 5×5 convolutions. section 11.2.2

From architecture to PyTorch

A naive Inception block defines parallel convolution, batch-normalization, and ReLU sequences. A dimensionality-reduced version inserts 1×1 convolutions before the 3×3 and 5×5 branches, reducing the channels processed by larger kernels.

Figure 11.8 Inception_v1 architecture

Use the architecture figure to compare the parallel branches and their multi-scale arrangement. section 11.2.2

How Math and Architectures of Deep Learning covers it

How deep it goes

Defines the multi-kernel idea and connects it to PyTorch branches and dimensionality reduction, without deriving convolutional cost formulas.

Examples

Uses close-up and distant cars, Inception_v1 architecture, and naive and dimensionality-reduced PyTorch blocks.

What it assumes

Assumes convolutional neural-network familiarity while motivating kernel size, depth, width, overfitting, and computational cost.

What you can do after

Builds multi-branch Inception-style blocks in PyTorch with parallel kernels and channel-reduction branches.

How it connects

Connects kernel size, object scale, convolution, dimensionality reduction, and Inception architecture.

liveBookmanning.com

Read Math and Architectures of Deep Learning on liveBook Get the book at manning.com

This explainer was generated by Manning's knowledge graph from the text of Math and Architectures of Deep Learning. The section links point to the parts of the book it draws on.