Chapter 11: Neural networks for image classification and object detection

Introduction

This introduction distinguishes image classification from object detection, explains why CNNs replaced hand-crafted vision features, and previews the chapter’s deep-learning architectures and applications.

• Classification and detection distinction: Distinguish image classification from object detection, including that classification assigns one or more predetermined object categories while detection also identifies each object’s location.
  Why it matters: This distinction determines whether a vision system needs only category outputs or both categories and locations.
• Bounding box representation: Describe a bounding box as an axis-parallel rectangle that specifies an object’s location using its top-left and bottom-right xy coordinates: [(xtl, ytl), (xbr, ybr)].
  Why it matters: Bounding boxes provide the concrete location output that turns recognition into object detection.
• CNN feature learning rationale: Explain why convolutional neural networks are preferable to traditional hand-crafted vision features and rules: CNNs learn relevant features from data, which is more scalable for images with variation in object appearance, scale, perspective, lighting, and background.
  Why it matters: Learning features from data is the central reason deep CNNs can address complex real-world vision problems.

Key terms: image classification, object detection, computer vision, bounding box, (xtl, ytl), (xbr, ybr), convolutional neural networks, CNNs, hand-crafted image features, feature learning

11.1 CNNs for image classification: LeNet

Introduces LeNet as a multi-convolutional-layer network that classifies resized MNIST digit images into one of ten classes.

• MNIST classification task: Describe the MNIST image-classification task used to introduce LeNet, including the input image, the ten possible digit labels, and the training and test data sets.
  Why it matters: This task makes concrete what an image classifier receives and what it is expected to predict.
• LeNet processing pipeline: Trace the LeNet pipeline from a resized 32 × 32 MNIST image through three 5 × 5, stride-1 convolutional layers, pooling after the first two layers, flattening, fully connected layers, and a ten-dimensional output.
  Why it matters: The pipeline is a foundational example of how convolutional features are converted into class predictions.
• Pooling and activation roles: Explain how pooling and nonlinear activations contribute to LeNet: 2 × 2 local-average pooling halves feature-map resolution and reduces sensitivity to input shifts and distortions, while tanh makes the network capable of nonlinear combinations of inputs.
  Why it matters: These operations explain why a stack of CNN layers can be both more robust to image variation and more expressive than an all-linear network.

Key terms: MNIST data set, training set, test set, image classification, label, LeNet, convolutional layer, 5 × 5 kernel, stride, feature map

11.1.1 PyTorch- Implementing LeNet for image classification on MNIST

11.2 Toward deeper neural networks

This section explains why deeper convolutional networks are needed for complex image classification and how stacks of small convolutions increase expressiveness while matching a larger receptive field with fewer parameters.

• Benefits of convolutional depth: Explain why LeNet's three convolutional layers can work for simple MNIST images but lack the expressive power needed for complex real-world image classification. Describe how additional convolutional layers add learnable parameters and nonlinearities, with lower layers learning simple features such as lines and edges and higher layers learning more abstract shapes or feature combinations.
  Why it matters: Depth lets a CNN build complex visual concepts from progressively simpler features.
• Small-filter receptive-field efficiency: Describe how sequential small convolution filters can give an output location the same receptive field as one larger filter while using fewer parameters. State the section's examples: one 5 × 5 convolution and two successive 3 × 3 convolutions both use a 5 × 5 receptive field, but require 25 versus 18 parameters for a single-channel input; likewise, one 7 × 7 filter and three 3 × 3 filters have the same receptive field, with the 7 × 7 filter having 81% more parameters.
  Why it matters: Using stacked small filters is a key reason deep CNNs can expand their visual reach without the parameter cost of large filters.
• Deep CNN architecture lineage: State that AlexNet, published by Krizhevsky and colleagues in 2012, reignited the deep-learning revolution by significantly outperforming prior state-of-the-art approaches on the ImageNet Large Scale Visual Recognition Challenge. Identify ImageNet in this example as a dataset of 1.3 million images in 1,000 classes, and name GoogleNet, VGG, and ResNet as subsequent deep image-classification networks mentioned in the section.
  Why it matters: This historical context situates the major CNN architectures introduced in the following subsections.

Key terms: expressive power, nonlinearity, learnable parameters, convolutional layers, receptive field, feature map, 5 × 5 convolution, 3 × 3 convolution, stride, zero padding

11.2.1 VGG (Visual Geometry Group) Net

This section explains VGG’s small-filter deep CNN design, its use of ReLU, and how its configurable convolutional backbone and classifier can be implemented in PyTorch.

• VGG small-filter design: Explain how VGG replaces the larger 7 × 7 or 11 × 11 filters used in earlier networks with 3 × 3 convolutions throughout a deeper network. State that stacking three 3 × 3 convolutions gives the receptive field of one 7 × 7 convolution while adding intervening nonlinearities and using fewer parameters.
  Why it matters: This design shows why depth built from small convolutional filters can be both more expressive and more parameter-efficient.
• VGG block and classifier structure: Describe the common VGG architecture: five convolutional blocks followed by three fully connected layers. Explain that convolutions within a block use 3 × 3 kernels, stride 1, same padding, and ReLU, while max pooling at each block end halves spatial resolution and the number of feature maps doubles from block to block.
  Why it matters: The repeated block pattern explains how VGG progressively converts an image into compact features for classification.
• ReLU and vanishing gradients: Explain why sigmoid and tanh can cause vanishing gradients in deep networks: their derivatives become small for high- or low-valued inputs and are multiplied across layers during backpropagation. Describe how ReLU has derivative 1 for positive inputs and 0 otherwise, avoiding this small-positive-gradient multiplication and enabling faster convergence.
  Why it matters: Understanding ReLU explains a key architectural choice that makes deep networks train more effectively.
• Configurable PyTorch VGG construction: Describe how the PyTorch implementation constructs VGG by using a reusable convolutional block and a configuration-driven convolutional backbone, then feeding that backbone’s output to a three-layer fully connected classifier. State that configurations distinguish variants such as VGG-11 and VGG-19 by specifying different numbers of convolutional layers, and that torchvision’s existing VGG implementation is recommended in practice.
  Why it matters: A configuration-driven backbone makes it practical to create and vary VGG architectures without rewriting the network structure.

Key terms: VGG, 3 × 3 convolution filters, receptive field, ReLU, expressive power, parameters, conv block, max pool layer, same padding, stride

11.2.2 Inception: Network-in-network paradigm

This section explains how Inception_v1 processes image features at multiple scales while controlling computation through parallel branches and 1×1 dimensionality reduction.

• Multi-scale Inception branches: Explain why an Inception module uses parallel 1×1, 3×3, and 5×5 convolutions plus a pooling branch instead of committing to one convolution kernel size. Describe how concatenating their outputs lets the network learn which scale-specific features to emphasize.
  Why it matters: Inception shows how an architecture can recognize objects at differing image scales without choosing one fixed receptive-field size.
• Dimensionality reduction bottlenecks: Describe the parameter-growth problem in a naive Inception block, especially for 5×5 convolutions and pooled outputs. Explain how inexpensive 1×1 convolutions before the 3×3 and 5×5 branches, and after pooling, reduce channel dimensions and computational cost.
  Why it matters: The 1×1 bottleneck is what makes multi-scale processing practical without an explosive number of parameters.
• Inception scaling rationale: Explain the two drawbacks of blindly increasing network depth or width—overfitting and greater training and inference computation—and state how Inception aims to increase depth and width within a constant computational budget.
  Why it matters: This rationale clarifies that architectural design is not only about adding capacity, but also about generalization and resource cost.
• GoogLeNet auxiliary classifiers: State that GoogLeNet stacks nine dimension-reduced Inception modules and uses global average pooling at the end. Describe how two intermediate auxiliary classifiers use softmax and auxiliary losses, combined as a weighted sum with the main loss, to help prevent the middle of a deep network from dying out because of vanishing gradients.
  Why it matters: Auxiliary classifiers illustrate one way the original GoogLeNet supported training in a very deep architecture.

Key terms: Inception module, kernel size, scale, 1 × 1 convolution, 3 × 3 convolution, 5 × 5 convolution, naive Inception block, dimensionality reduction, input channels, output features

11.2.3 ResNet: Why stacking layers to add depth does not scale

This section explains why simply stacking more layers can worsen a network and how ResNet residual blocks and skip connections make very deep CNNs practical.

• Depth degradation problem: Explain the degradation problem observed when a 56-layer network has higher training and test error than a 20-layer network. Distinguish it from overfitting and from vanishing or exploding gradients in the ResNet example.
  Why it matters: This explains why greater theoretical expressive power does not automatically make a deeper network easier to train.
• Residual learning formulation: Describe how a residual block models a target function h(x) as h(x)=g(x)+x, where g(x)=h(x)-x is the residual learned by the stacked layers. Explain why this makes an identity mapping easy to represent when g(x)=0.
  Why it matters: Residual learning gives deep networks an easy fallback to passing information through unchanged while still allowing them to learn useful features.
• Shortcut connection benefits: Justify shortcut connections as parameter-free paths that make identity behavior easy and provide a direct route for information and gradients between lower and higher layers. State that residual blocks can still learn non-identity feature transformations when those are useful.
  Why it matters: The shortcut is not merely an architectural decoration: it improves optimization without restricting what the feature-learning layers can do.
• ResNet block structure: Describe the high-level ResNet architecture: an initial 7×7 stride-2 convolution and max-pooling stage, stacked residual convolutional blocks using mostly 3×3 filters, and average pooling followed by a fully connected classifier. Explain that ResNet downsamples with stride-2 convolutions and doubles feature channels when feature-map spatial size is halved.
  Why it matters: This structural pattern explains how ResNet builds increasingly abstract image features while controlling per-layer computation.
• Dimension-changing shortcuts: Explain the two shortcut cases in a residual convolutional block: direct addition when input and output have the same spatial resolution and feature count, and a 1×1 stride-2 convolution on the shortcut when spatial resolution is halved and output features are doubled.
  Why it matters: The shortcut and residual outputs must have matching shapes before they can be added, so projection shortcuts enable ResNet stage transitions.

Key terms: degradation problem, training error, test error, overfitting, vanishing gradients, exploding gradients, residual learning, residual, identity function, shortcut connection

11.2.4 PyTorch Lightning

This section shows how PyTorch Lightning organizes and automates end-to-end LeNet training and inference for MNIST while leaving model research logic under the developer’s control.

• Lightning boilerplate automation: Explain why the section uses PyTorch Lightning instead of vanilla PyTorch for LeNet training. Describe the training infrastructure Lightning automates while the developer retains the research logic.
  Why it matters: Lightning lets you focus on model experiments rather than repeatedly writing training-system plumbing.
• Lightning training components: Describe the three main components used to train a model with PyTorch Lightning: DataModule, LightningModule, and Trainer. Explain the responsibility and separation provided by each component.
  Why it matters: This division lets data, model logic, and training orchestration be reused and changed independently.
• LightningModule implementation hooks: Describe what a LightningModule contains for the LeNet classifier, including the model definition, training_step, validation_step, testing_step, and configure_optimizers. State that Lightning invokes these hooks at the appropriate points in training.
  Why it matters: Knowing the required hooks identifies where to put model-specific behavior without writing the entire execution loop.
• Trainer execution and inference: Describe how the Trainer runs model fitting through trainer.fit rather than a hand-written trainer loop, and how inference is performed with model.predict. State the automatic logging benefit for viewing loss and accuracy curves in TensorBoard.
  Why it matters: These high-level calls provide a concise path from a configured model to training monitoring and prediction.

Key terms: PyTorch Lightning, boilerplate code, research logic, multi-GPU training, floating-point 16, TPU, DataModule, LightningDataModule, LightningModule, Trainer

11.3 Object detection: A brief history

This section distinguishes object detection from image classification and introduces the region-based classification idea behind R-CNN.

• Object detection task: Explain why image classification alone may not adequately describe an image containing multiple objects, and describe object detection/localization as identifying each object's category and bounding-box coordinates.
  Why it matters: Object detection must answer both what objects are present and where they appear in the image.
• R-CNN region classification: Describe the basic R-CNN strategy: extract image regions intended to contain one object each, classify every region with an image-classifier deep neural network, and select the regions with the highest confidence.
  Why it matters: This region-proposal-then-classification pattern provides the historical starting point for the object detectors developed next.

Key terms: image classification, object detection/localization problem, object categories, bounding-box coordinates, region-based CNN, R-CNN, regions, image classifier deep neural network, confidence

11.3.1 R-CNN

This section explains the original R-CNN object-detection pipeline, from proposing candidate regions through CNN feature extraction to classification and box refinement.

• R-CNN detection pipeline: Describe the three stages of R-CNN: selective search proposes regions of interest, a deep convolutional neural network extracts a feature vector for each region, and class-specific classifiers and bounding-box regressors produce an object class and refined location.
  Why it matters: R-CNN established the region-based pipeline from which later object detectors evolved.
• Fixed-size region processing: Explain why R-CNN warps each proposed region to a fixed size before passing it to the deep neural network: candidate regions can have arbitrary sizes, whereas the network typically expects fixed-sized inputs.
  Why it matters: This input-size conversion is an important practical consequence of applying a CNN separately to variable-sized image regions.
• Region training labels: Describe how R-CNN labels training regions using their overlap with ground-truth boxes: a region with high overlap receives a positive ground-truth class label, and one without high overlap receives a negative label.
  Why it matters: The detector needs labeled candidate regions so that its classifiers can learn which proposals correspond to real objects.

Key terms: R-CNN, selective search, regions of interest, region proposals, feature extraction, classification, fixed-sized inputs, warped, deep convolution neural network, ground-truth (GT) class

11.3.2 Fast R-CNN

This section explains how Fast R-CNN makes R-CNN object detection much faster and more end-to-end by sharing CNN feature extraction across region proposals and jointly performing classification and bounding-box regression.

• R-CNN computational bottlenecks: Describe why the original R-CNN approach is slow and cumbersome: it runs feature extraction independently for every region proposal and uses a multistage pipeline with selective search, a deep network, SVMs, and bounding-box regressors.
  Why it matters: These limitations explain exactly what Fast R-CNN was designed to improve.
• RoI pooling feature sharing: Explain how RoI pooling lets Fast R-CNN process the whole image through the CNN once and then use region-of-interest bounding boxes on the CNN output to obtain features for all proposed regions.
  Why it matters: Shared image features are the central reason Fast R-CNN avoids thousands of redundant forward passes.
• End-to-end multitask detection: Trace the Fast R-CNN detection flow: selective search generates roughly 2,000 RoIs, RoI pooling extracts their features from one CNN pass, and classification and bounding-box regression heads jointly classify and localize objects. Explain that multitask loss removes the need for separate SVMs and makes the architecture more end-to-end.
  Why it matters: Combining classification and localization in one trained network simplifies the detector and improves object-detection quality as well as speed.
• Fast R-CNN speed improvements: State that shared feature extraction makes Fast R-CNN substantially faster than R-CNN—reported as 9 times faster in training and 213 times faster at test time—and that the section says it also improves detection quality.
  Why it matters: The reported gains quantify the practical impact of avoiding repeated per-region CNN computation.

Key terms: R-CNN, region proposal, forward pass, selective search, SVM, bounding-box regressor, Fast R-CNN, RoI pooling, region of interest, RoI

11.3.3 Faster R-CNN

This section introduces Faster R-CNN as an object detector that replaces CPU-based selective search with a learned region proposal network while sharing convolutional features with Fast R-CNN processing.

• Selective-search bottleneck: Explain why Fast R-CNN still had a speed limitation: it relied on selective search to generate region proposals, and selective search runs only on CPUs and is slow and time-consuming.
  Why it matters: Faster R-CNN was designed specifically to remove the proposal-generation bottleneck left in Fast R-CNN.
• Learned proposal generation: Describe Faster R-CNN's central change: a deep network learns to generate region proposals instead of using selective search. State that this made it one of the first near-real-time detectors and produced better proposals, improving overall accuracy.
  Why it matters: Learning proposals inside the deep architecture removes an external bottleneck and improves the quality of the detector's candidate regions.
• Two-module shared architecture: Describe Faster R-CNN as an RPN plus an R-CNN module: the RPN efficiently predicts proposals across a range of scales and aspect ratios, while the R-CNN module applies RoI pooling followed by classification and regression. Explain that both modules share the same convolutional layers and weights rather than learning separate convolutional networks.
  Why it matters: The division of work and shared convolutional features are the architectural basis for Faster R-CNN's efficient detection pipeline.

Key terms: Fast R-CNN, selective search, region proposals, CPU, Faster R-CNN, FRCNN, near-real-time object detectors, region proposal network, RPN, R-CNN module

11.4 Faster R-CNN: A deep dive

This section introduces the end-to-end Faster R-CNN architecture, tracing how an input image becomes object classes and bounding boxes.

• Faster R-CNN processing pipeline: Trace the high-level Faster R-CNN pipeline from an input image through a convolutional backbone, region proposal network, RoI pooling, and fully connected layers to object predictions.
  Why it matters: This pipeline provides the map for understanding the individual Faster R-CNN components that follow.
• Detection output structure: Describe the predictions made for each candidate region: softmax probabilities over K object classes plus a background class, and four bounding-box coordinate values for each of the K classes.
  Why it matters: Object detection must both decide what is present and specify where it is located.

Key terms: Faster R-CNN, convolutional backbone, feature maps, region proposal network, RPN, regions of interest, softmax probability estimates, K object classes, background class, bounding-box coordinates

11.4.1 Convolutional backbone

This subsection explains how Faster R-CNN uses a modified VGG-16 convolutional backbone to produce shared lower-resolution feature maps for its detection modules.

• Modified VGG-16 backbone: Describe the convolutional backbone used in the original Faster R-CNN implementation, including that VGG-16 convolution layers are shared by both the RPN and R-CNN modules and that the final pooling layer after conv5 is removed.
  Why it matters: The shared backbone provides the feature maps on which Faster R-CNN performs both region proposals and region classification.
• Backbone feature-map resolution: Trace how removing VGG-16's last pooling layer leaves four spatial halvings, reducing image dimensions by a factor of 2^4 = 16; state the resulting 14 × 14 map for a 224 × 224 input and 50 × 50 map for an 800 × 800 input.
  Why it matters: Feature-map resolution determines the spatial detail available to later detection stages while reducing the amount of computation.

Key terms: FRCNN, VGG-16, convolutional backbone, RPN, R-CNN, conv5, feature map, max pooling, spatial size, conv block

11.4.2 Region proposal network

This section explains how a Faster R-CNN region proposal network generates, trains, and filters anchor-based object proposals for an image.

• Anchor-based RPN architecture: Describe how the RPN operates over a backbone convolutional feature map using multiple anchors at every sliding-window location. Explain that its fully convolutional heads predict an objectness score and bounding-box offsets for each anchor.
  Why it matters: The RPN converts shared image features into candidate object regions that the rest of Faster R-CNN can classify.
• Anchor diversity and translation invariance: Explain why anchors span multiple scales and aspect ratios and why this divides detection into simpler shape-specific tasks. Describe how shared convolutional weights let the same anchor configuration detect a similarly sized object at different image locations.
  Why it matters: Anchors make the varied sizes and shapes of real objects manageable without requiring one detector head to solve every case at once.
• IoU anchor target assignment: Explain how intersection over union measures overlap between an anchor and a ground-truth box, and state the RPN rules for positive, negative, and ignored anchors. Include that positive anchors are the highest-IoU anchor(s) for a ground-truth box or have IoU above 0.7, while non-positive anchors below 0.3 for every ground-truth box are negative.
  Why it matters: Correct target assignment tells the RPN which anchors should learn objectness and localization and which represent background.
• Balanced anchor minibatches: Describe the severe positive-negative anchor imbalance and how Faster R-CNN addresses it by undersampling negatives. State that loss is computed from 256 randomly sampled anchors per image with up to a 1:1 positive-to-negative ratio, padding with negatives when fewer than 128 positives are available.
  Why it matters: Balancing the sampled anchors prevents a misleading high-accuracy solution that labels nearly every anchor as background.
• Regression targets and RPN loss: Explain that only positive anchors receive bounding-box regression targets, expressed as offsets from the anchor to the ground-truth box. Describe the combined loss: cross-entropy objectness loss for positive and negative sampled anchors, plus smooth L1 regression loss for positive anchors only.
  Why it matters: The two-part objective teaches the RPN both to recognize likely objects and to move reference anchors onto their object boxes.
• Proposal selection and suppression: Trace how RPN predictions become region proposals: convert predicted offsets back to boxes, clip boxes to the image, discard boxes below the minimum size, rank candidates by objectness, and apply non-maximal suppression. Explain that NMS removes highly overlapping redundant boxes, using a 0.7 threshold before retaining the top RoIs.
  Why it matters: Proposal filtering reduces thousands of redundant anchor predictions to a useful set of candidate regions for the detector.

Key terms: region proposal network, RPN, convolutional feature map, anchor, objectness, box-regression layer, anchor configurations, scale, aspect ratio, sliding window

11.4.3 Fast R-CNN

This section explains how the Fast R-CNN stage turns RPN proposals into object classes and refined bounding boxes using RoI pooling, supervised training, and class-specific post-processing.

• RoI pooling mechanism: Explain how RoI pooling extracts each proposed region’s features from a shared convolutional feature map and converts variable-sized RoI feature maps into a fixed H-by-W output using regional max pooling.
  Why it matters: RoI pooling makes it possible to classify and localize many proposed regions without running the convolutional network separately for each one.
• Fast R-CNN output head: Describe the Fast R-CNN head after RoI pooling: fixed-length RoI features pass through fully connected layers and branch into a softmax classifier and a class-specific bounding-box regressor.
  Why it matters: The two output branches jointly answer the two detection questions: what object is present and where its box should be.
• RoI supervision and sampling: Describe how Fast R-CNN assigns labels and regression targets to RPN RoIs using highest-IoU ground-truth matches, and how it samples a balanced training minibatch.
  Why it matters: The detector needs both meaningful positive and background examples and box targets only where an object is actually present.
• Multitask detection loss: Explain that Fast R-CNN training combines cross-entropy classification loss for RoIs with smooth L1 box-regression loss only for non-background RoIs.
  Why it matters: This loss trains the network to recognize categories while refining locations without asking background regions to predict nonexistent boxes.
• Class-specific detection inference: Trace Fast R-CNN inference from each RoI’s class probabilities and class-specific offsets through box conversion, clipping, and per-class nonmaximum suppression.
  Why it matters: These post-processing steps turn overlapping proposal-level predictions into final labeled boxes within the image.

Key terms: RoI pooling, conv feature map, RoI, subsampling factor, max pooling, RoIAlign, FC layers, classifier, regressor, softmax

11.4.4 Training the Faster R-CNN

This section explains why Faster R-CNN trains its RPN and Fast R-CNN components with a shared convolutional backbone, comparing alternate optimization with faster approximate joint training.

• Shared backbone training rationale: Explain why training the RPN and Fast R-CNN as completely independent networks is undesirable and why they share convolutional layers. State that separate training is expensive because it creates two independently modified backbones, whereas a shared backbone is more efficient and sensible because both modules need similar convolutional features.
  Why it matters: Shared features make Faster R-CNN efficient while supporting both proposal generation and object detection.
• Alternate optimization sequence: Describe alternate optimization for Faster R-CNN: train the RPN first, use its proposals to train Fast R-CNN, then use the Fast R-CNN-tuned network to initialize the RPN and repeat the alternation over multiple training rounds.
  Why it matters: Alternate optimization is the original way to coordinate training of the two Faster R-CNN subnetworks while sharing features.
• Approximate joint training tradeoff: Describe approximate joint training as merging the RPN and Fast R-CNN during training, generating proposals in each SGD forward pass, treating those proposals as fixed for Fast R-CNN training, and backpropagating the combined losses. Explain that it is approximate because the proposals actually depend on the RPN, so one derivative is ignored; it is preferred because it is much faster while achieving similar accuracy to alternate optimization.
  Why it matters: This explains why Faster R-CNN can be trained end to end efficiently despite proposal generation depending on learned network outputs.

Key terms: RPN, Fast R-CNN, convolutional layers, convolutional backbone, shared convolutional layers, alternate optimization, AltOpt, region proposals, initialize, approximate joint training

11.4.5 Other object-detection paradigms

This section compares YOLO, SSD, and FPN as object-detection paradigms in terms of their architectures and speed–accuracy trade-offs relative to Faster R-CNN.

• YOLO single-stage detection: Describe how YOLO detects objects with a single convolutional network that predicts multiple bounding boxes and their class probabilities directly from the full image. Distinguish its one-stage, fast design from Faster R-CNN’s RPN-plus-Fast-R-CNN two-stage pipeline and state the associated accuracy trade-off.
  Why it matters: YOLO illustrates how removing proposal-based stages can make object detection fast enough for real-time use, while sacrificing some accuracy.
• SSD multiscale predictions: Explain how SSD makes single-stage predictions of class scores and box offsets for a fixed set of default bounding boxes using small convolutional filters on feature maps. Describe how its anchor-based, multiscale feature-map predictions seek a balance between YOLO’s speed and Faster R-CNN’s accuracy.
  Why it matters: SSD shows how a detector can retain a simple proposal-free architecture while using anchors and multiple feature-map scales to improve accuracy.
• FPN feature fusion: Describe the feature-map pyramid in convolutional networks: deeper maps have lower spatial resolution but stronger semantic information, while earlier high-resolution maps contain lower-level features. Explain how FPN combines these maps through a top-down upsampling path and lateral 1 × 1-convolution connections so that merged maps at every level can make predictions, particularly improving small-object detection.
  Why it matters: FPN demonstrates that combining semantic context with high-resolution detail can improve detection of objects that would be poorly represented on only the final low-resolution feature map.
• Detection architecture trade-offs: Compare the section’s speed–accuracy positioning of the three paradigms: YOLO is the fastest but less accurate, SSD improves on YOLO’s accuracy while remaining below Faster R-CNN especially for small objects, and FPN on Faster R-CNN is more accurate but slower than YOLO/SSD-style approaches.
  Why it matters: Choosing an object detector requires matching the architecture’s speed and detection-quality trade-off to the application.

Key terms: YOLO, single-stage detector, bounding boxes, class probabilities, full image, two-stage detector, SSD, single-stage network, default bounding boxes, anchors

11.5 Summary

This summary recaps CNN architectures for increasingly capable image classification and the two-stage Faster R-CNN pipeline for object detection and localization.