Data discovery turns a clearly defined business problem into a feasible data plan. Identify the datasets needed, confirm that stakeholders can access them, and decide which tables to use. This happens before cleaning, exploratory data analysis, and modeling; discovery and preprocessing can together take 60%–70% of project time. section 1.4 section 11.3
Check feasibility before committing to a model:
- List relevant sources and decide how to host and share them. Set appropriate permissions for the people who need access. section 1.4 section 11.3
- Check that the data is complete, relevant, sufficiently representative, and large enough for the business problem. Consider whether its time span captures important variation. section 11.3
- Plan how the data will be refreshed after deployment. Flag missing values, outliers, and questions about business rules for closer examination during preparation. section 11.3
Retail or telecom data
A retail or telecom project may need at least a year of data to capture seasonality, alongside variables for sales, transactions, discounts, purchases, marketing, and historical behavior. Finding files is not enough: their coverage and accessibility must also support the intended analysis. section 11.3
