A convolutional layer extracts local patterns rather than combining every input value with every output. A small kernel slides across an input, computes a weighted sum over each local neighborhood, and emits a feature map. Reusing the same kernel weights at every position greatly reduces parameters compared with a fully connected layer. In image models, early layers can detect edges and corners; later stacked layers can combine those features into progressively higher-level structures. section 10.0 section 10.8
For one-dimensional input, picture a ruler sliding along a rope. At each stop, multiply aligned input and kernel values and sum the products. Kernel size determines the local window, stride determines how far the kernel moves between stops, and padding controls what happens at the boundaries. Valid padding stops when the kernel would extend outside the input; zero padding treats out-of-range values as zero. Algebraically, the operation is a moving weighted local sum and can be represented as a structured matrix multiplication, so familiar forward propagation and backpropagation apply. section 10.1 section 10.1.3 section 10.8
Smoothing and detecting edges
Kernel weights determine which local property responds strongly. Uniform weights average a neighborhood, smoothing a noisy curve or image. Positive and negative weights can suppress uniform regions while producing a high response where values change sharply, revealing curve edges or horizontal and vertical image edges. For video, a 3D kernel slides across time, height, and width; comparing local averages in successive frames yields high responses in moving regions. section 10.1.2 section 10.3.1 section 10.3.2 section 10.4.1
In a trained neural network, kernel weights are usually not hand-selected. Configure layer dimensions such as channels, kernel size, stride, and padding; initialize weights; then let loss minimization and backpropagation learn them. A 2D layer receives PyTorch tensors shaped N × C × H × W, where N is batch size, C channels, and H and W image dimensions. Each Conv2d layer produces output channels, and layers can be chained with nonlinearities and pooling. section 10.3.3 section 10.6 section 10.6.1
Listing 10.8 PyTorch code for a sample convolutional neural network
import torch
class SampleCNN(torch.nn.Module):
def __init__(self, num_classes):
super(LeNet, self).__init__()
self.nn = torch.nn.Sequential( #1
torch.nn.Conv2d(
in_channels=1, out_channels=6,
kernel_size=5, stride=1), #2
...
torch.nn.Conv2d(
in_channels=6, out_channels=16,
kernel_size=5, stride=1),
...
torch.nn.Conv2d(
in_channels=16, out_channels=120,
kernel_size=5, stride=1), #3
...
)
def forward(self, x): #4
out = self.nn(x)
return out
Inspect a compact CNN whose Conv2d layers increase channels from 1 to 6, then 16, then 120 while chaining layers in a Sequential model. section 10.6.1
Stacking convolutional layers expands what each output can incorporate. Two 3 × 3 convolutions can reach the same 5 × 5 receptive field as one 5 × 5 convolution while using fewer parameters and adding another nonlinearity. Pooling commonly follows convolution to downsample feature maps and reduce sensitivity to small shifts and distortions. LeNet applies three 5 × 5 convolutional layers, pooling after the first two, and activation layers for handwritten-digit classification. section 11.2 section 10.7 section 11.1
