Downsampling reduces a feature map’s spatial resolution while concentrating information into fewer locations and more channels. It makes later features represent larger regions of the original input and limits the information lost during final aggregation. section 8.1.2 section 11.3.2
Common implementation choices differ according to whether spatial location must be preserved:
- Max pooling extracts windows and keeps each channel’s maximum, commonly using a 2 × 2 window with stride 2 to halve spatial dimensions. section 8.1.2
- Strided convolution downsamples while applying a learned transformation, retaining more information about where features occur than max pooling. section 11.3.2
- Classification can tolerate max pooling, but segmentation needs location-aware downsampling because each output must correspond to an input pixel. section 11.3.2
Three downsampling steps in segmentation
A segmentation encoder uses convolutions with stride 2 three times, reducing an image to activations of size (25, 25, 256). Transposed convolutions then increase resolution until predictions match the original 200 × 200 image dimensions. section 11.3.2
Listing 8.5 Incorrectly structured convnet missing max-pooling layers
inputs <- keras_input(shape = c(28, 28, 1)) outputs <- inputs |> layer_conv_2d(filters = 64, kernel_size = 3, activation = "relu") |> layer_conv_2d(filters = 128, kernel_size = 3, activation = "relu") |> layer_conv_2d(filters = 256, kernel_size = 3, activation = "relu") |> layer_global_average_pooling_2d() |> layer_dense(10, activation = "softmax") model_no_max_pool <- keras_model(inputs = inputs, outputs = outputs)
Inspect a classification model that omits max-pooling layers: its feature maps remain large, limiting spatial hierarchy and leaving excessive information for global averaging. section 8.1.2
get_model <- function(img_size, num_classes) {
conv <- function(..., padding = "same", activation = "relu") #1
layer_conv_2d(..., padding = padding, activation = activation)
conv_transpose <- function(..., padding = "same", activation = "relu")
layer_conv_2d_transpose(..., padding = padding, activation = activation)
input <- keras_input(shape = c(img_size, 3))
output <- input |>
layer_rescaling(scale = 1/255) |> #2
conv(64, 3, strides = 2) |>
conv(64, 3) |>
conv(128, 3, strides = 2) |>
conv(128, 3) |>
conv(256, 3, strides = 2) |>
conv(256, 3) |>
conv_transpose(256, 3) |>
conv_transpose(256, 3, strides = 2) |>
conv_transpose(128, 3) |>
conv_transpose(128, 3, strides = 2) |>
conv_transpose(64, 3) |>
conv_transpose(64, 3, strides = 2) |>
conv(num_classes, 3, activation = "softmax") #3
keras_model(input, output)
}
model <- get_model(img_size = img_size, num_classes = 3)
model
Model: "functional"
┌─────────────────────────────────┬────────────────────────┬───────────────┐
│ Layer (type) │ Output Shape │ Param # │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ input_layer (InputLayer) │ (None, 200, 200, 3) │ 0 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ rescaling (Rescaling) │ (None, 200, 200, 3) │ 0 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ conv2d (Conv2D) │ (None, 100, 100, 64) │ 1,792 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ conv2d_1 (Conv2D) │ (None, 100, 100, 64) │ 36,928 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ conv2d_2 (Conv2D) │ (None, 50, 50, 128) │ 73,856 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ conv2d_3 (Conv2D) │ (None, 50, 50, 128) │ 147,584 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ conv2d_4 (Conv2D) │ (None, 25, 25, 256) │ 295,168 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ conv2d_5 (Conv2D) │ (None, 25, 25, 256) │ 590,080 │
└─────────────────────────────────┴────────────────────────┴───────────────┘
┌─────────────────────────────────┬────────────────────────┬───────────────┐
│ conv2d_transpose │ (None, 25, 25, 256) │ 590,080 │
│ (Conv2DTranspose) │ │ │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ conv2d_transpose_1 │ (None, 50, 50, 256) │ 590,080 │
│ (Conv2DTranspose) │ │ │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ conv2d_transpose_2 │ (None, 50, 50, 128) │ 295,040 │
│ (Conv2DTranspose) │ │ │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ conv2d_transpose_3 │ (None, 100, 100, 128) │ 147,584 │
│ (Conv2DTranspose) │ │ │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ conv2d_transpose_4 │ (None, 100, 100, 64) │ 73,792 │
│ (Conv2DTranspose) │ │ │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ conv2d_transpose_5 │ (None, 200, 200, 64) │ 36,928 │
│ (Conv2DTranspose) │ │ │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ conv2d_6 (Conv2D) │ (None, 200, 200, 3) │ 1,731 │
└─────────────────────────────────┴────────────────────────┴───────────────┘
Total params: 2,880,643 (10.99 MB)
Trainable params: 2,880,643 (10.99 MB)
Non-trainable params: 0 (0.00 B)
Inspect a complete R/Keras segmentation model that alternates ordinary and strided convolutions before reconstructing resolution with transposed convolutions. section 11.3.2
