Data quality is the degree to which data is accurate, complete, consistent, timely, and fit for its intended purpose. In AI products, it shapes both model performance and user trust: weak labels can degrade classifiers, while noisy, biased, stale, or sensitive training data can produce unreliable language-model outputs. section 4.2.1 section 5.1.1
Treat data quality as an operational practice rather than a one-time cleanup task:
- Define annotation guidelines and quality criteria, then check label agreement across annotators and revise the process when agreement is low. section 4.2.1
- Watch for imbalanced classes and insufficient training-data size, since both can bias or weaken classification results. section 4.2.1
- Account for human-labeling risks such as boredom, fatigue, and subjective bias; complex or domain-specific labeling may require substantial human oversight. section 4.2.1
- Validate generated content when source data may contain misinformation, and use human review or retrieval-augmented generation where factual accuracy matters. section 5.1.1
- Check whether the model’s knowledge is current enough, supplementing frozen training data with real-time sources when relevance depends on new information. section 5.1.1
A recommendation and content-product contrast
For fashion recommendations, collecting user ratings during product operation can improve the training data, but incentives are needed to gather enough feedback. For a content-generation product, user review alone may not catch subtle bias or factual errors, so output validation and human review remain necessary. section 4.2.1 section 5.1.1
Table 4.2 A sample of labeled examples
| Features |
Label |
|||||
| user_id |
purchased_items |
purchase_value |
last_active |
n_visits |
search_queries |
segment |
| abhj3k |
2 |
908 |
2024-04-30 08:36:24 |
48 |
3 |
Seekers |
| shj67d |
0 |
0 |
2023-12-26 12:56:24 |
24 |
45 |
Conservatives |
| i963gh |
12 |
673 |
2024-05-15 23:22:11 |
156 |
25 |
Indecisives |
| ty54df |
20 |
1250 |
2024-05-10 14:21:07 |
190 |
5 |
Champions |
A labeled-example table makes data quality concrete by showing the feature fields and segment labels that support classifier training and review. section 4.2.1
