Fast R-CNN is a two-stage object-detection architecture that avoids running a convolutional network separately for every proposed region. A convolutional backbone processes the whole image once, producing a feature map. Region proposals are then mapped onto that shared map, converted into fixed-size features, and passed to heads that classify objects and refine their bounding boxes. This replaces the older pipeline of per-region CNN passes, separate SVMs, and bounding-box regressors with a more end-to-end detector. section 11.3.2 section 11.4.3
Its detection path has four main steps:
- Generate candidate regions, typically about 2,000 RoIs per image with selective search. section 11.3.2
- Run the full image through convolutional and pooling layers once to create a convolutional feature map. section 11.3.2 section 11.4.3
- Map each RoI from image coordinates to feature-map coordinates and use RoI pooling to obtain a fixed-size representation despite differing RoI sizes. section 11.4.3
- Feed each pooled representation to classification and bounding-box regression heads, then select predicted labels, probabilities, and class-specific offsets at inference. section 11.3.2 section 11.4.3
RoI pooling is the central efficiency idea. If the backbone has a subsampling factor of 16, an image RoI from (0, 0) to (256, 256) corresponds to the (0, 0) to (16, 16) area of the convolutional feature map. The feature-map embeddings already summarize receptive fields in that image area. RoI pooling divides the mapped region into bins and max-pools each bin, yielding the same output dimensions for large and small RoIs. The classifier can therefore consume a fixed-length vector for every proposal. section 11.4.3
Figure 11.18 A conv feature map with two regions of interest of different sizes. RoI pooling extracts a fixed-sized output feature map (2 × 2 in this image) from each of the RoIs in a single pass via max pooling. This enables us to extract fixed-sized representative feature vectors for each RoI, which are then fed into further classifier and regressor layers for classification and localization.
Compare differently sized RoIs on one shared feature map: max pooling produces equally sized feature maps that can feed the same classifier and regressor. section 11.4.3
Training combines object classification with localization. The RoI head produces scores for object classes plus background and predicts four box offsets for each class. Classification uses cross-entropy loss; localization uses Smooth L1 loss on positive RoIs after selecting the offsets for each RoI’s target class. The combined objective lets one network learn recognition and box refinement together. section 11.3.2 section 11.4.3
From image region to pooled feature
With a 16× subsampling factor, a 256×256 image region maps to a 16×16 region of the convolutional feature map. Rather than resize that image crop and run another CNN pass, Fast R-CNN pools the mapped feature region into a fixed grid, then classifies it and predicts bounding-box corrections. section 11.4.3
Fast R-CNN substantially improves on R-CNN because all RoIs share one expensive convolutional computation: reported speedups are 9× during training and 213× at test time. Its remaining bottleneck is selective search, which still generates proposals on the CPU. Faster R-CNN retains the Fast R-CNN RoI classification-and-regression module but replaces selective search with a learned region proposal network that shares convolutional layers with it. section 11.3.2 section 11.3.3
