Data collection obtains the instances needed for analysis or machine learning. After defining the task, inputs, and targets, it often becomes the most time-consuming and costly project activity. Dataset quality—especially size, label reliability, and feature quality—strongly affects generalization. section 6.1.2
Choose a collection strategy that matches the available signals and the prediction task:
- Reuse existing behavioral records when they already represent the target, such as user likes for music recommendation or historical click-through rates for advertising. section 6.1.2
- Acquire or create labeled data when the required signal is unavailable, such as tagging social media posts for spam detection or images for visual classification. section 6.1.2
- Plan for domain expertise and varied conditions when collecting specialized data, such as archaeological site databases paired with satellite images from different weather conditions. section 6.1.2
Improve a dataset before investing heavily in model development:
- Collect enough points to densely sample the input–output space. section 5.4.1
- Inspect inputs for anomalies, proofread labels, clean the data, and address missing values. section 5.4.1
- Use feature selection when many features are available but their usefulness is uncertain, and apply feature engineering where it can improve generalization. section 5.4.1
- Treat additional data collection as a potentially higher-return investment than incremental model improvements. section 6.1.2 section 5.4.1
“DEWEY DEFEATS TRUMAN”: a famous example of sampling bias
Use the sampling-bias example to see how the way data is collected can misrepresent the population and undermine conclusions. section 6.1.2
