A data source is where the data for an ML system comes from. A purchase history, a third-party database, and labels supplied by users are different sources, even if their contents eventually share one dataset. Start with the system’s goal: it determines which sources are useful. Availability and quality also depend on the business and its existing systems. section 6.1
When looking for data, check more than the most convenient internal database:
- Recorded activity and physical processes may supply trades and prices, satellite images, or sensor readings. section 6.1
- Business operations and external databases may supply purchase histories or domain-specific data collected by another company. section 6.1
- Expert teams or end users may produce labels. Simulators, rendered environments, and generative AI may produce artificial data. section 6.1
Do not assume one source is enough. Compare what the business already has with what it could obtain externally. Unique data can be a competitive advantage, but easier-to-acquire data can also be valuable when prepared well. Record each source in the dataset design, alongside how data will be represented and stored, filtered, transformed into features, and labeled. Record the source of the labels separately. section 6.7 section 6.8
A successful job with no data
Suppose a system personalizes promotional offers using a customer database, clickstream logs, and call center logs merged into a warehouse. If an upstream source stops supplying data, the whole system can suffer. An ETL job might even report success after processing zero rows, leaving the problem unnoticed for a week. A green job status is therefore not proof that the expected data arrived. section 14.3.1
Keep the origin of data consistent as the system moves from development to use. Training data gathered one way and production inputs gathered another can create a distribution mismatch that harms performance and fairness. The system’s own outputs may also change the behavior that generates future data. Define origins, preprocessing, and filtering clearly; maintain metadata; and check the pipeline for reproducibility, consistency, and availability. section 6.6 section 6.8
