AlexNet’s 2012 image-classification breakthrough put five convolutional layers at the center of automatic feature extraction. Rather than rely on handcrafted image features, these layers learn filters that respond to local patterns. Early layers can pick up edges and textures; later ones combine simpler responses into shapes and object parts. section 2.5.1
A filter’s size sets the local area it examines, while its stride sets how far it moves at each step. These choices affect both the detail retained and the spatial size of the resulting feature maps. Because parameters are shared across positions, a convolutional layer’s neuron count is less straightforward than its parameter count. section 2.5.1
Early downsampling in AlexNet
AlexNet begins with a 224 × 224 × 3 RGB image. Its first convolution uses 11 × 11 filters with a stride of 4, reducing the spatial dimensions to 55 × 55. Subsequent max pooling reduces them further. section 2.5.1
Figure 2.3 The AlexNet architecture (cropped in the original paper). The numbers indicate spatial dimensions and channel depth at each stage. The input image (224 × 224 × 3 RGB) passes through a series of convolutional and pooling layers that progressively reduce spatial resolution (224→55→27→13) while increasing channel depth. Early downsampling is driven by the first convolutional layer (11 × 11 filters with a stride of 4), followed by max pooling layers (3 × 3 windows with a stride of 2) that further reduce spatial dimensions. The stride determines how far a filter moves across the input at each step, controlling how densely the image is sampled and how quickly spatial resolution is reduced. Filter sizes (11, 5, 3) are depicted as cubes and define the local receptive field of each convolution. The two horizontal streams reflect the model’s original two-GPU training setup, where feature maps were split across devices (e.g., 48 + 48 = 96 channels). The fully connected layers (4096 neurons, implemented as 2048 + 2048 across GPUs) feed into a 1000-way output for ImageNet classification. Used with permission granted by the lead author (Alex Krizhevsky).
Follow the changing spatial dimensions and channel depth through AlexNet’s convolutional and pooling layers; the two streams also show its original split across GPUs. section 2.5.1
Convolution was once treated as an indispensable architectural ingredient. Transformers later showed that attention could replace convolution and recurrence in the setting that motivated their design—not that convolution had ceased to be useful. section 5.8
