SAM (Segment Anything) generates pixel-level segmentation masks from an image and a prompt, such as a point inside an object or a box around it. It can segment objects outside a fixed class list without fine-tuning first. section 11.4 section 11.5
Segmenting a mango with a box
Provide the box coordinates for the mango’s top-left and bottom-right corners, then pass the image and box to the pretrained model. Displaying the returned mask produces a segmented mango. section 11.4.5
py_require("keras-hub") #1
keras_hub <- import("keras_hub") #2
model <- keras_hub$models$ImageSegmenter$from_preset("sam_huge_sa1b")
Load the pretrained SAM model through KerasHub’s ImageSegmenter task, avoiding an implementation from scratch. section 11.4.1
SAM combines several components:
- An image encoder converts the input image into a smaller image embedding. section 11.4.2
- A prompt encoder converts point or box prompts into an embedded vector. section 11.4.2
- A mask decoder combines both embeddings and produces several possible masks. section 11.4.2
Training uses image, prompt, and mask triples. The SA-1B dataset contains fully segmented images, with about 100 masks per image on average and sometimes more than 500; this makes SAM useful for rapidly creating annotated segmentation datasets. section 11.4.2 section 11.4.5
