A Region Proposal Network (RPN) is the first stage of Faster R-CNN: it replaces selective search with a learned mechanism that proposes rectangular regions likely to contain objects. It works alongside the detection stage, which receives those regions, applies RoI pooling, and predicts object classes and refined bounding boxes. Sharing one convolutional backbone makes the two-stage design more efficient than maintaining separate feature extractors. section 11.3.3 section 11.4
Figure 11.12 architecture. (Source: “Faster R-CNN: Toward real-time proposal networks”; https://arxiv.org/abs/1506.01497.)
The architecture separates feature extraction, proposal generation, RoI pooling, and final classification and box regression. The RPN consumes backbone feature maps rather than raw image patches independently. section 11.4
For a VGG backbone, an image of size (h, w) produces a feature map of roughly (h/16, w/16). Each feature-map location corresponds spatially to a 16 × 16 input patch, while its learned representation has a larger receptive field. A small fully convolutional network slides across this map: a 3 × 3 convolution produces a 512-dimensional feature vector at each location, then parallel 1 × 1 heads predict objectness and bounding-box offsets. section 11.4.2 section 11.5
Anchors make detection across varied object shapes manageable. At every feature-map location, the RPN places reference boxes over multiple scales and aspect ratios. Rather than directly discovering every possible box shape, each anchor specializes the prediction task: the network decides whether that anchor is object-like and predicts how to move and resize it. With anchor center (xa, ya) and size (wa, ha), a target box is encoded as tx = (x − xa)/wa, ty = (y − ya)/ha, tw = log(w/wa), and th = log(h/ha). section 11.4.2 section 11.4.2
Generating anchors on an 800 × 800 image
Using a subsampling factor of 16 creates a 50 × 50 feature-map grid. At each grid center, anchor generation combines configured scales and aspect ratios. For each combination, width is computed as subsample × scale × √aspect_ratio and height as subsample × scale × √(1/aspect_ratio), then converted into top-left and bottom-right coordinates. Concatenating anchors across all grid locations yields the candidate reference boxes for the image. section 11.4.2
Training and proposal generation follow a focused sequence:
- Discard anchors outside the image, compare valid anchors with ground-truth boxes using intersection over union (IoU), and assign training labels. section 11.4.2
- Train objectness with cross-entropy while ignoring designated anchors; train offsets only for positive anchors with Smooth L1 loss, weighted with the classification loss. section 11.4.2
- Decode predicted offsets into boxes, clip them to image bounds, remove boxes below a minimum size, and rank remaining candidates by objectness. section 11.4.2
- Apply non-maximal suppression (NMS): retain the highest-scoring box, remove remaining boxes whose IoU exceeds a threshold, and repeat. section 11.4.2
The RPN and detector can be trained by alternate optimization: train the RPN, use its proposals to train Fast R-CNN, then iterate while transferring the shared backbone. Approximate joint training instead merges them, combines their losses, and backpropagates in one process. It is faster, but approximate because RPN-generated proposals are treated as fixed during detector training even though they depend on RPN outputs. section 11.4.4 section 11.5
