An ML system needs a dataset to function, but collecting many records is not enough. Start with the prediction goal, then decide what each example represents, where its data will come from, and whether the resulting pipeline can supply it reliably. section 6.0 section 6.4 section 6.8
Choose sources to fit the goal rather than assuming one source will suffice:
- Look inside the business for transactions and other recorded operations; consider external databases, physical measurements, or broader activity data when relevant. section 6.1
- When existing records lack the needed examples or labels, consider expert or user labeling and synthetic data. section 6.1
- Keep metadata alongside data so flows remain consistent. section 6.8
How much is enough? More examples help only when they add information relevant to the problem. Duplicates and heavy noise may add little value. Active learning can target cases the model gets wrong or handles with low confidence. For a huge, imbalanced dataset, stratified sampling can limit overrepresented groups; if timing matters, newer records may deserve priority. section 6.4
Sales forecasting
For retail sales forecasting, one example can be a (date, product, store) bundle, with units sold as the target. Transaction history supplies sales records, while stock history helps distinguish weak demand from products that were unavailable. Product and store metadata, planned promotions, competitor prices, and weather can add context. The design choice is not merely which tables exist, but which sources bear on the prediction. section 6.7.1
Starting before production data exists
A hard-hat detector may need images from customer cameras that cannot be accessed before deployment. Possible starting points include rendered images, relevant public images, manually staged photos, or a simple baseline that helps begin collecting real data. None is automatically suitable: the substitute data must be close enough to the intended setting to be useful. section 6.5
Figure 6.3 A plot showcasing model metric improvements over dataset size for a real project. Each line demonstrates the dynamics of the target metric for a single task once the dataset size is growing.
Compare target-metric trajectories as dataset size grows. Use measured improvement, not record count alone, to judge whether collecting more examples is worthwhile. section 6.4
Finish by checking the operating path, not just the saved dataset: data must be available before the system runs, and the pipeline must be reproducible, consistent, and available. The preparation sequence depends on the business goal and domain. section 6.8
