A mask marks which parts of data belong to a selected region or group. What happens to those parts depends on the task: an image mask can restrict where edits happen, a segmentation mask can identify an object’s pixels, and a Boolean mask can select samples for a calculation. section 2.2 section 15.4.1 section 13.6.1
For inpainting, picture a digital stencil laid over a photograph. White marks the area a diffusion model may repaint; black protects the rest. The image supplies the starting scene, the text prompt describes the desired change, and the mask says where that change may occur. Original content outside the mask is reinserted during denoising. section 2.2 section 2.2.1
Repaint a horse, not its background
Pass a horse image, a separately prepared mask image, and a prompt asking for a zebra in the same pose and lighting to StableDiffusionInpaintPipeline. The mask limits the requested replacement to the selected area. The pipeline is used as a pretrained tool; its components are not reimplemented for this example. section 2.2.2
For classification metrics, a mask selects samples rather than pixels. Comparing labels and predicted scores with a 0.5 threshold produces negative-label and negative-prediction Boolean masks. Inverting them produces positive masks. These selections let you count correct predictions and average loss separately for each class. The simple inversions rely on the problem having two classes; multiclass or multilabel data needs different logic. section 13.6.1
Listing 13.17 Masking for classification metrics
negLabel_mask = metrics_t[METRICS_LABEL_NDX] <= classificationThreshold negPred_mask = metrics_t[METRICS_PRED_NDX] <= classificationThreshold posLabel_mask = ~negLabel_mask posPred_mask = ~negPred_mask
The comparisons create negative-class masks, and `~` creates their positive-class counterparts. section 13.6.1
In segmentation, a mask describes an image region pixel by pixel. A generated binary mask uses True for pixels belonging to an object and False for those outside it. Segment Anything can generate multiple masks for an image and assign confidence scores; its mask decoder combines image features and prompt information to produce them. section 15.4.1 section 15.4
Outline a suspected nodule
A CT dataset may provide a point marking a candidate nodule when training needs a pixel-level mask. Use the CT slice and point to create a mask, then pair images with masks for segmentation training. After fine-tuning SegFormer, compare its predicted masks on unseen slices with the reference masks. section 15.6.2 section 15.8 section 15.9
