A box prompt is an approximate rectangle around an object that guides an image-segmentation model toward the object’s mask. The rectangle does not need to be precise. section 11.4.4
Segment Anything combines the image embedding with a prompt embedding, then produces several candidate masks. Training creates box prompts by drawing a rectangle around every point inside a selected ground-truth mask. section 11.4.2
Creating a training box
Given an image and a selected object mask, collect all pixels labeled as part of the object and draw the smallest rectangle enclosing them. Pair the image, rectangle, and mask as one training example. section 11.4.2
A box prompt fits into a broader prompting workflow:
- Use a box when an approximate object boundary is easier to provide than a precise pixel mask. section 11.4.4
- Combine prompting with Segment Anything’s predicted masks, which can include multiple alternatives ranked by predicted quality. section 11.4.4
- Treat point prompts as an alternative input, especially when foreground and background points can clarify an ambiguous object. section 11.4.4
