Image segmentation

As explained in Inside Deep Learning: Math, Algorithms, Models

A computer vision task that assigns a label to each pixel in an image so that object regions are separated from the background or each other.

Image segmentation assigns a class to every pixel instead of producing one label for an entire image. A 200 × 200 image therefore requires 40,000 pixel classifications. Labels commonly represent objects such as people, horses, cars, or background; the resulting regions can support object detection or precise measurements such as tumor-cell proportions. section 8.0 section 8.1

A practical PyTorch segmentation pipeline follows these steps:

• Load the image and its masks, remove unused alpha channels, move channels first, normalize pixel values, and resize inputs to 256 × 256. section 8.1.1
• Merge separate nucleus masks with a logical OR to form one binary target: 1 for object pixels and 0 for background. section 8.1.1
• Use convolutional layers to preserve spatial outputs; for two classes, produce one logit per pixel and train with BCEWithLogitsLoss. section 8.1.2
• Compare fine-detail convolutional models with pooled and transposed-convolution models before combining both resolutions with U-Net skip connections. section 8.3 section 8.3.1

Nuclei segmentation

For the 2018 Data Science Bowl images, a Dataset returns a normalized three-channel microscope image and a single-channel binary mask. A U-Net can then produce output shaped like (B, 1, W, H), predicting whether each pixel contains a nucleus. section 8.1.1 section 8.3.1

Figure 8.1 Example of an input image (on the left) with the segmented ground-truth labels (on the right) from the PASCAL VOC 2012 dataset. Every pixel has been given a class, with a default “no class” or “background” class given to pixels that are not part of a labeled object.

Use the labeled image pair to inspect how every pixel receives an object or background class in a PASCAL VOC example. section 8.1

How Inside Deep Learning: Math, Algorithms, Models covers it

How deep it goes

Descends from pixel-wise classification through dataset preparation, loss selection, convolutional architectures, and nested U-Net implementation.

Examples

Uses PASCAL VOC object labels, Data Science Bowl nuclei masks, and medical examples involving tumor-cell proportions.

What it assumes

Assumes image-classification and CNN foundations while teaching pixel targets, binary outputs, and segmentation-specific data preparation.

What you can do after

Build it from scratch using a PyTorch Dataset, binary masks, pixel-wise loss, convolutional models, and nested U-Net blocks.

How it connects

Connects image segmentation to object detection, convolutional neural networks, max pooling, transposed convolutions, skip connections, and binary cross-entropy.

liveBookmanning.com

Read Inside Deep Learning: Math, Algorithms, Models on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Deep Learning with PyTorch, Second EditionA learning task that partitions an input, often an image, into labeled regions or elements, used here to flag voxels or pixels that may belong to a nodule.
• 

  liveBookmanning.com
  Deep Learning with R, Third EditionImage segmentation is the task of assigning semantic labels to individual pixels or regions of an image.
• 

  liveBookmanning.com
  Machine Learning System DesignSegmentation is a computer-vision labeling formulation that identifies the pixels belonging to an object or region.
• 

  liveBookmanning.com
  Deep Learning with Python, Third EditionA vision task that assigns a label to each pixel or region of an image, providing denser localization than object detection.
• 

  liveBookmanning.com
  Sutskever's ListImage segmentation assigns semantic or instance labels to image pixels to delineate objects and regions.

Compare how 6 books explain image segmentation →

Related concepts

• classifiercontrasts with · A classifier is a predictive computational model that assigns inputs to one or more discrete classes or categories.
• object detectioncontrasts with · Object detection is a computer-vision task that identifies object instances in visual data, assigns them categories, and estimates their locations.
• computer visionpart of · Computer vision is the field of computing concerned with using computational methods to extract and interpret information from images, video, and other visual data.

This explainer was generated by Manning's knowledge graph from the text of Inside Deep Learning: Math, Algorithms, Models. The section links point to the parts of the book it draws on.