Image segmentation answers “where?” rather than just “is it there?” A classifier might report that an image contains a cat; a segmentation model produces an image-like mask showing which pixels belong to the cat. For a CT scan, the same idea flags pixels or voxels that might belong to a lung nodule. section 15.3 section 15.1
Finding possible nodules is a first pass, not a diagnosis. Many flagged structures will not be nodules. Segmentation tells a separate classifier where to look; classification then examines candidates more closely. Here the two models are trained separately rather than optimized together end to end. section 15.1 section 11.4.1
There are several ways to mark what matters in an image:
- Semantic segmentation assigns a class to each pixel. A binary nodule mask marks candidate pixels as true and other tissue as false; it does not distinguish one nodule from another. section 15.2
- Instance segmentation gives separate labels to individual objects, such as two distinct tumors. section 15.2
- Object detection places bounding boxes around objects rather than labeling their pixels precisely. section 15.2
A practical nodule workflow uses 2D CT slices so masks are easier to inspect than they would be in 3D. The dataset supplies slices and corresponding nodule masks. Segment Anything (SAM) demonstrates mask generation guided by a point prompt. To find candidates on unseen slices without supplying a point, a pretrained SegFormer semantic-segmentation model is fine-tuned on the CT images and masks, with background and nodule as its two labels. section 15.1 section 15.7.1 section 15.9
Checking a predicted mask
Pass an unseen CT slice through the trained model, post-process its output into a segmentation map, and display the slice beside the annotated mask and prediction. This makes it possible to see which suspected nodule pixels the model marked and where its prediction differs from the annotation. section 15.8
Figure 15.17 CT images (left) with the ground truth mask (middle) and the model predictions (right)
Compare the CT image, ground-truth mask, and predicted mask side by side. section 15.8
