Data quality is the degree to which data is accurate, complete, consistent, relevant, timely, and fit for its intended use. In an ML project, survey the data early to locate issues that may require investigation, resolution, or avoidance. section 5.2.4
Turn the quality survey into concrete checks:
- Inspect representative records for useful information, ambiguity, and patterns that may support the intended application. section 5.2.4
- Count empty records, NULL values, and placeholder values such as “.” or “n/a” that conceal missing mandatory fields. section 5.2.4
- Assess whether apparent regularities are useful, expected, irrelevant, or signals of unusable data. section 5.2.4
Smart-building incident records
Review incident descriptions and resolutions for actionable content. Reports about HVAC failures may relate naturally to unusual sensor readings, but extracting structured facts such as affected floors can be difficult. Inspect more than a handful of records before deciding whether the data is suitable. section 5.2.4
select count(*) from incidents; >1781 select * from incidents limit 1; >23-05-2021, CRITICAL, 360, "HVAC failure…", "affinity engineer..")
Use the incident count and sample-record query as a compact starting point for a data-quality survey. section 5.2.4
