Data mining discovers possible patterns or relationships by collecting data first and forming hypotheses from what appears in it. It can reveal useful questions, but open-ended searching also increases the risk of spurious findings. section 1.7 section 2.2
Use data mining as an exploratory process, then examine the result carefully:
- Collect or obtain data, recognizing that machine learning may proceed directly to fitting a model. section 1.7
- Inspect the data for patterns, relationships, or other useful signals before committing to a specific hypothesis. section 1.7
- Fit a model or other analysis to the apparent pattern, then test it on data it has not seen. section 1.7
- Question the data’s context, representativeness, missing information, and possible bias before treating the finding as reliable. section 2.2
Photographs and golf ability
Suppose photographs are collected to decide whether someone is good at golf. A fist-pump may celebrate another player, and a scorecard may be forged. Narrowing the objective and collecting relevant evidence is safer than mining an unrestricted family-photo archive. Searching repeatedly for a favorable statistical result can become p-hacking: selecting models or data because they produce a desired low p-value rather than a realistic conclusion. section 2.2 section 9.12
Figure 1.1 An example of the four steps in applying statistical analysis to data, studying whether temperature affects sports drink sales
Trace the four-stage analysis of whether temperature affects sports-drink sales: data, hypothesis, model fitting, and testing. section 1.7
