Non-maximum suppression

As explained in Math and Architectures of Deep Learning

A post-processing method that removes redundant overlapping detections while keeping the highest-scoring ones.

Non-maximum suppression (NMS) de-duplicates object detections. Region proposal networks produce many boxes for the same object because nearby anchors independently predict it. Keeping every overlapping proposal would create a redundant set of regions of interest (RoIs), so NMS retains strong candidates and suppresses highly overlapping alternatives. section 11.4.2

Given bounding boxes, confidence scores, and an overlap threshold, NMS repeatedly selects the highest-scoring remaining box, adds it to the output, and compares it with the boxes left. Any remaining box whose intersection over union (IoU) with the selected box exceeds the threshold is removed. The result is a filtered set of boxes rather than many near-duplicates. section 11.4.2

A typical region-proposal pipeline applies NMS after earlier filtering steps:

• Convert predicted offsets into bounding-box coordinates, then clip boxes to image boundaries. section 11.4.2
• Discard boxes smaller than a minimum height or width, score the survivors by objectness, and sort them. section 11.4.2
• Keep a pre-NMS candidate set, run NMS with a 0.7 threshold, then retain the top post-NMS RoIs. section 11.4.2

From overlapping anchors to a compact RoI set

In an RPN, anchors are sampled at a stride of 16 pixels, so one reasonably sized object may be predicted by multiple anchors. NMS at a 0.7 threshold suppresses highly overlapping proposals. The illustrated result reduces 24 bounding boxes before NMS to 4 afterward, leaving a more useful RoI set for the R-CNN. section 11.4.2

NMS is also used after Fast R-CNN produces class scores and class-specific box offsets. At this stage it runs separately within each class: overlapping boxes for the same class compete, while overlapping boxes assigned to different classes are not suppressed merely for overlapping. section 11.4.3

In PyTorch, `torchvision.ops.nms(rois, objectness_scores, nms_threshold)` returns indices of retained RoIs. Those indices can then select the post-NMS boxes, with separate limits for training and testing. section 11.4.2

Figure 11.17 The left column shows the bounding boxes (24) before NMS. The right column shows the bounding boxes (4) that remain after NMS.

Compare the dense pre-NMS proposals with the four boxes retained after suppression to see how overlap-based de-duplication compresses detections. section 11.4.2

How Math and Architectures of Deep Learning covers it

How deep it goes

Defines the score-ordered IoU suppression procedure and shows PyTorch invocation, without deriving IoU.

Examples

Illustrates overlapping stride-16 anchor predictions and reduces 24 pre-NMS boxes to 4 retained boxes.

What it assumes

Assumes familiarity with bounding boxes, confidence scores, and IoU while giving the suppression procedure directly.

What you can do after

Apply it via `torchvision.ops.nms` to filter scored RoIs using an overlap threshold.

How it connects

Connects bounding boxes, objectness scores, IoU, anchors, region proposals, RoIs, and Fast R-CNN.

liveBookmanning.com

Read Math and Architectures of Deep Learning on liveBook Get the book at manning.com

Related concepts

• Intersection over Unionuses · Intersection over Union is the ratio of the intersection area or volume of two regions to the area or volume of their union.
• bounding boxaddresses · A bounding box is an axis-aligned rectangular or hyperrectangular region defined by coordinate limits that encloses or localizes an object, area, or set of points.
• thresholduses · A threshold is a boundary value against which a quantity is compared to determine whether a condition is met or an action or decision is triggered.
• anchor boxaddresses · A preset bounding box template with a specific scale and aspect ratio used as a reference for object localization.

This explainer was generated by Manning's knowledge graph from the text of Math and Architectures of Deep Learning. The section links point to the parts of the book it draws on.