Pooling in Convolutional Neural Networks

Pooling is a fundamental operation in convolutional neural networks (CNNs) that serves to downsample feature maps, thereby reducing their spatial dimensions. This process is essential for retaining the most significant features while simultaneously decreasing computational complexity and achieving local translation invariance. Pooling enhances the network’s robustness to variations in the position of features within the input data. The two primary types of pooling operations are max pooling and average pooling.

Max Pooling

Max pooling is one of the most common pooling techniques used in CNNs. It involves dividing the input feature map into non-overlapping tiles, usually of size 2x2, and then taking the maximum value from each tile to form the downsampled output. This method is particularly effective in preserving the most prominent features detected by the convolutional layers.

Figure 8.8 Max pooling in detail.

The intuition behind max pooling is that the output images from a convolution layer, especially after an activation function, tend to have high magnitudes where specific features, such as vertical lines, are detected. By retaining the highest value in each 2x2 neighborhood, max pooling ensures that these significant features survive the downsampling process, even if it means discarding weaker responses.

Figure 10.19 Max pooling using a 2 × 2 kernel with stride 2.

Average Pooling

Average pooling, on the other hand, computes the average of the values within the defined region of the input feature map. This method also reduces the size of the feature map but tends to preserve more contextual information compared to max pooling.

Figure 10.20 Average pooling using a 2 × 2 kernel with stride 2.

Combining Convolutions and Downsampling

Pooling is often used in conjunction with convolutional operations to recognize larger structures within an image. As illustrated in Figure 8.9, a set of 3x3 kernels is first applied to an 8x8 image, resulting in a multichannel output image of the same size. This output is then downsampled by half, producing a 4x4 image. A second set of 3x3 kernels is applied to this downsampled image, effectively mapping back to 8x8 neighborhoods of the original input.

Figure 8.9 More convolutions by hand, showing the effect of stacking convolutions and downsampling.

This process allows the second set of kernels to take the output of the first set (features like averages, edges, etc.) and extract additional features on top of those. By stacking convolutions and downsampling, CNNs can effectively capture and highlight complex structures within the input data.

Implementation in PyTorch

The following PyTorch code snippet demonstrates how to implement both max pooling and average pooling:

import torch

X = torch.tensor([                #1
    [0, 12, 26, 39],
    [6, 19, 31, 44],
    [12, 25, 38, 50],
    [18, 31, 43, 57]
], dtype=torch.float32).unsqueeze(0).unsqueeze(0)

max_pool_2d = torch.nn.MaxPool2d( #2
    kernel_size=2, stride=2)

out_max_pool = max_pool_2d(X)     #3

avg_pool_2d = torch.nn.AvgPool2d( #4
    kernel_size=2, stride=2)

out_avg_pool = avg_pool_2d(X)     #5
#1 Instantiates a 4 × 4 input tensor
#2 Instantiates a 2 × 2 max pooling layer with stride 2
#3 Output feature map is of size 2 × 2
#4 Instantiates a 2 × 2 average pooling layer with stride 2
#5 Output feature map is of size 2 × 2

In this code, a 4 × 4 input tensor is processed through both max pooling and average pooling layers, each with a 2 × 2 kernel and a stride of 2. The resulting output feature maps are reduced to a size of 2 × 2, demonstrating the downsampling effect of pooling operations.