Image segmentation
Image segmentation is a computer-vision task that partitions an image and assigns labels to its pixels or regions to delineate objects or meaningful areas.
Deep Learning with PyTorch, Second Edition
A learning task that partitions an input, often an image, into labeled regions or elements, used here to flag voxels or pixels that may belong to a nodule.
- How it is used
- Finds potential lung nodules in CT slices so a separate classifier knows where to look.
- How deep it goes
- Explains mask prediction and model components, then walks through dataset preparation, fine-tuning, and inference.
- Examples
- Uses cat-versus-background masks and lung CT nodule masks to illustrate pixel-level labeling.
- What you can do after
- Apply it via SegFormer to generate and visualize nodule masks for unseen CT slices.
Read how Deep Learning with PyTorch, Second Edition explains image segmentation →
Inside Deep Learning: Math, Algorithms, Models
A computer vision task that assigns a label to each pixel in an image so that object regions are separated from the background or each other.
- How it is used
- Uses pixel-level classification to locate object regions and measure their extent rather than predict one class per image.
- How deep it goes
- Works through binary-mask preparation, PyTorch loss selection, and competing convolutional architectures, ending with a nested U-Net implementation.
- Examples
- Uses multiclass object labels and a 2018 Data Science Bowl nuclei dataset with individual masks merged into a binary target.
- What you can do after
- Build it from scratch in PyTorch using prepared masks, a binary loss, and a nested U-Net for per-pixel predictions.
Read how Inside Deep Learning: Math, Algorithms, Models explains image segmentation →
Deep Learning with R, Third Edition
Image segmentation is the task of assigning semantic labels to individual pixels or regions of an image.
- How it is used
- Frames computer vision problems that require pixel-level masks rather than whole-image labels or bounding boxes.
- How deep it goes
- Explains masks, encoder-decoder architecture, spatial downsampling, training, metrics, checkpointing, and visualization.
- Examples
- Uses custom-background video calls, an R/Keras three-class model, predicted and target masks, and Segment Anything prompts.
- What you can do after
- Build it from scratch with strided Conv2D and Conv2DTranspose layers, or apply Segment Anything with point and box prompts.
Read how Deep Learning with R, Third Edition explains image segmentation →
Machine Learning System Design
Segmentation is a computer-vision labeling formulation that identifies the pixels belonging to an object or region.
- How it is used
- Frames image segmentation as a product and data-labeling choice when exact pixels matter.
- How deep it goes
- Defines pixel-level labeling and illustrates subpixel precision, soft labels, and annotation constraints without detailing model architectures.
- Examples
- Uses assembly-line component inspection, medical malignancy labeling, and hair-background separation as examples.
- What you can do after
- Evaluate whether pixel-level labels fit product needs and whether annotation resources support them.
Read how Machine Learning System Design explains image segmentation →
Fight Fraud with Machine Learning
Image segmentation is the task of assigning labels to pixels or regions of an image.
- How it is used
- Applies pixel-level labeling to locate forged regions in receipt images.
- How deep it goes
- Defines segmentation, contrasts classification, names DeepLabV3, and demonstrates inference without fully detailing model training internals.
- Examples
- Uses a segmented beach photograph and receipt forgery probability heatmaps compared with bounding-box labels.
- What you can do after
- Apply it via a CNN to generate interpretable pixel-level forgery predictions and inspect their generalization on unseen receipts.
Read how Fight Fraud with Machine Learning explains image segmentation →
Deep Learning with Python, Third Edition
A vision task that assigns a label to each pixel or region of an image, providing denser localization than object detection.
- How it is used
- Deploys pixel-level masks for image editing, autonomous driving, robotics, medical imaging, and video applications.
- How deep it goes
- Explains three segmentation types, dataset preparation, and encoder-decoder construction, while deferring detailed model internals.
- Examples
- Uses video-call background replacement, cat-and-dog masks, the Oxford-IIIT Pets dataset, and Segment Anything prompting.
- What you can do after
- Builds a segmentation dataset and model from scratch or applies Segment Anything with image, text, point, or box prompts.
Read how Deep Learning with Python, Third Edition explains image segmentation →
Sutskever's List
Image segmentation assigns semantic or instance labels to image pixels to delineate objects and regions.
- How it is used
- Use pixel-wise labels to delineate objects and surfaces where preserving spatial detail matters.
- How deep it goes
- Contrasts segmentation with classification and explains how dilated convolutions expand context, without detailing a full model implementation.
- Examples
- Illustrates pixel labels with roads, cars, and dogs, and compares segmentation predictions against ground truth.
- What you can do after
- Understand why dense prediction needs spatial detail and how dilated convolutions widen context without reducing resolution.
Also covered in
What to know first
- computer visionComputer vision is the field of computing concerned with extracting and interpreting information from images, video, and other visual data.
- language modelA language model is a computational model that represents patterns in linguistic sequences to assign probabilities, predict, generate, or evaluate language.
- pixelA pixel is either a discrete image sample carrying visual information at a raster location or, in CSS, an abstract reference unit for rendered dimensions.
- classifierA classifier is a predictive model that assigns an input to one or more discrete classes or categories.
Related concepts
- classifiercontrasts with · A classifier is a predictive model that assigns an input to one or more discrete classes or categories.
- object detectioncontrasts with · Object detection is a computer-vision task that identifies object instances in visual data, assigns them categories, and estimates their locations.
- semantic segmentationincludes · Semantic segmentation is the computer-vision task of assigning a semantic class label to each image pixel without distinguishing individual instances of the same class.
- computer visionpart of · Computer vision is the field of computing concerned with extracting and interpreting information from images, video, and other visual data.
- DeepLabV3uses · DeepLabV3 is a convolutional-neural-network-based image segmentation architecture that uses atrous convolution and spatial pyramid pooling to capture information at multiple scales.
Questions about image segmentation
What is image segmentation?
Image segmentation is a computer-vision task that partitions an image and assigns labels to its pixels or regions to delineate objects or meaningful areas.
Which Manning books explain image segmentation?
Deep Learning with PyTorch, Second Edition, Inside Deep Learning: Math, Algorithms, Models, Deep Learning with R, Third Edition, Machine Learning System Design, Fight Fraud with Machine Learning, Deep Learning with Python, Third Edition and Sutskever's List.
What should I know before learning image segmentation?
Manning books that teach image segmentation usually introduce computer vision, language model, pixel and classifier first.
What is the difference between image segmentation and classifier?
Image segmentation is a computer-vision task that partitions an image and assigns labels to its pixels or regions to delineate objects or meaningful areas. A classifier is a predictive model that assigns an input to one or more discrete classes or categories.
What kinds of image segmentation are there?
Kinds of image segmentation include semantic segmentation and instance segmentation.
What is image segmentation part of?
Image segmentation is part of computer vision. Computer vision is the field of computing concerned with extracting and interpreting information from images, video, and other visual data.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.






