Data annotation adds labels or other structured learning signals to raw data. Labels express the desired outcome, such as a movie review’s sentiment or an image’s animal, and determine which machine-learning approaches are available. Most applications need labeled data, making annotation a skilled and often costly activity. section 3.1.2
Annotation quality directly affects classifier quality. Treat labeling as an operational process rather than a one-time clerical task: formalize the task, provide guidelines and tools, measure consistency, and review human factors such as boredom, fatigue, and subjective bias. section 4.2.1
Use a practical annotation checklist:
- Define accurate, consistent labeling rules and quality criteria before scaling the work. section 4.2.1
- Compare labels from multiple annotators on part of the dataset; investigate low agreement and revise the strategy or guidelines. section 4.2.1
- Check class balance and training-set size so scarce categories do not leave the classifier biased toward larger classes. section 4.2.1
- Choose the labeling route deliberately: use language models for simpler tasks, add human oversight for complex or domain-specific tasks, or collect feedback during product operation. section 4.2.1
Labels in practice
A training table can pair user features—purchased items, purchase value, visits, and search queries—with a segment label such as Seekers, Conservatives, Indecisives, or Champions. The resulting rows become labeled examples for classification, provided the labels are defined consistently and checked for balance. section 4.2.1
Annotation can also be generated automatically from the task itself. Language-model training derives labels from the next word in preceding text, while few-shot prompting can use curated examples or, for some predictive tasks, automatically assigned labels. The latter retains the label space and output format but should not be confused with reliable human ground truth. section 3.1.2 section 6.2.2
