liveBook · Manning

Context module

As explained in Sutskever's List

A context module is a neural-network component that aggregates multiscale spatial context, commonly by stacking convolutions with increasing dilation rates.

Dense prediction needs broad context without losing pixel-level detail. A context module stacks convolutions with increasing dilation rates, such as 1, 2, 4, and 8, to expand the receptive field while preserving full resolution. In Yu and Koltun’s 2015 approach, this helped address the resolution trade-off in ResNet-like architectures. section 3.5

The design connects receptive field, resolution, and model complexity:

  • Dilation inserts gaps between kernel elements, enlarging the receptive field without increasing the parameter count per layer. section 3.5
  • The segmentation model modified VGG-16 by removing pooling and striding layers and replacing them with dilated convolutions. On PASCAL VOC, it exceeded leading models by more than five percentage points in mean pixel IoU. section 3.5
Figure 3.6 Dilated convolutions rapidly expand the receptive field without increasing computational cost. Each dot represents a convolutional kernel element. As dilation increases (from left to right), the receptive field expands exponentially, capturing broader spatial context while keeping the parameter count the same. Used with permission granted by the lead author (Fisher Yu).
figure

See how increasing dilation expands the receptive field while keeping the convolution kernel elements—and parameter count—unchanged. section 3.5

How Sutskever's List covers it

How deep it goes
Explains dilation, receptive-field growth, parameter scaling, and a VGG-16 segmentation adaptation.
Examples
Uses dilation rates 1, 2, 4, and 8, plus a VGG-16 adaptation evaluated on PASCAL VOC.
What you can do after
Understand how stacked dilated convolutions gather broader context without sacrificing spatial resolution.
How it connects
Links dilated convolution and receptive fields to ResNet, VGG-16, and semantic segmentation.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Sutskever's List. The section links point to the parts of the book it draws on.