Data testing checks whether the data moving through a pipeline has the expected quality and properties. A pipeline job can run successfully yet deliver bad data: collection problems, data-entry mistakes, or faults in the infrastructure can change what reaches model development. Test the data as well as the pipeline code. section 4.3.3
Start during the data survey. Inspect completeness, coverage, quality, possible bias or poisoning, and labeling accuracy. Record integrity questions and results so the survey becomes a starting point for repeatable tests. In sprint 1, design and implement those tests alongside the pipelines that deliver data to the modeling team. section 5.2 section 5.2.5 section 5.4
Choose checks against plausible failures:
- Detect duplicate records, including those caused by replayed source data or copying errors. section 5.4.3
- Set reasonable bounds and minimums for the volume of data arriving. section 5.4.3
- Check input preconditions: expected columns, non-null counts, and distributions within expected bounds. section 5.4.3
- Check output postconditions and verify that excluded items stay excluded and features are constructed correctly from source data. section 5.4.3 section 4.3.3
Turn an expectation into a check
If a sensor is off, its reading should be null. Test that relationship in the pipeline input rather than relying on the pipeline job merely completing. A separate statistical check can flag data whose standard deviation falls outside an expected bound. section 5.4.3
Deploy tests strategically: large data sets and cross-comparisons can make them slow and expensive. Keep them in place as pipelines move toward production. Reusing pipeline code does not guarantee identical data; changes to feature creation can introduce new bias even when the model artifact is unchanged. Compare actual pipeline output with expected behavior to protect model fidelity. section 5.4.3 section 9.4.1
