Data annotation turns collected examples into targets a supervised-learning model can predict. Labels may be retrieved automatically, but many projects require people to label examples by hand, making annotation labor-intensive and central to model quality. section 6.1.2
Choosing labelers for image data
A cat-versus-dog classifier may need only general annotators. Dog-breed classification requires specialized knowledge, while labeling CT scans for bone fractures requires medical expertise. Crowdsourcing platforms such as Mechanical Turk can be inexpensive and scalable, but may produce noisy labels; specialized providers can save time while reducing control. section 6.1.2
Use these practical rules when planning annotation:
- Treat annotation quality as target quality: inconsistent labels limit the quality of the trained model. section 6.1.2
- Match the labeling workforce to the task: use broad participation for simple categories and experts for specialized judgments. section 6.1.2
- Consider control, cost, scalability, and noise when comparing self-labeling, crowdsourcing, and specialized companies. section 6.1.2
Human correction in image segmentation
For the SA-1B dataset, human experts first manually segmented a small image set. An initial model then assisted semiautomated collection, while people corrected and further annotated its results. The goal was a unique segmentation mask for every object, with roughly 100 masks per image on average. section 11.3.2
Figure 11.7 An example image from the SA-1B dataset
Inspect an example SA-1B image to see the many object masks produced through the annotation pipeline. section 11.3.2
