Data leakage occurs when label-related information enters the input features through an unrealistic artifact, often during data preparation. It inflates apparent performance and can taint every model built from the data. section 12.2
Investigate suspiciously strong performance by:
- Questioning whether an innocuous feature, such as length, correlates unexpectedly with the goal. section 12.2
- Thinking like a cheater and searching for shortcuts that reveal the label. section 12.2
- Labeling data by hand and comparing human shortcuts with the model’s apparent biases. section 12.2
- Fixing the leakage before continuing model development. section 12.2
Malware detection
A classifier reached near-perfect accuracy because benign Windows data commonly contained “Copyright Microsoft Corporation,” while malware data did not. The string exposed the benign label instead of representing a naturally valid predictive signal. section 12.2











