Completeness is the degree to which a dataset contains the records, attributes, places, and time periods needed for a decision. It is not a single pass/fail property: a model can be complete enough to count customers while lacking enough detail for customer segmentation, or a traffic dataset can have populated columns but uneven location and date coverage. section 3.4.4 section 8.4.1
Start by stating what must be present for the intended analysis. Inspect missing values by column, then decide whether absence is expected or erroneous. Missing road names may be legitimate for some road types, whereas a small number of absent traffic measurements may be treated as zeros when that assumption is acceptable. Record the decision rather than silently transforming the data. section 8.4.1
traffic.isnull().sum()
Count nulls per column to identify attributes whose absence needs explanation before analysis. section 8.4.1
Check record completeness separately from attribute completeness. Customer IDs appearing in purchase data without a matching customer record create rows with an identifier but no customer details. Dropping them can make records look more complete while understating the customer base. Keep such trade-offs visible and limit conclusions to tasks the available data can support. section 3.4.4
For time series, assess completeness across several dimensions:
- Compare regions and measurement locations to establish geographic coverage. section 8.4.1
- Calculate each location's earliest and latest measurement dates to compare duration of coverage. section 8.4.1
- Identify locations with only one day of data and compare their prevalence across regions. section 8.4.1
- Look for missing years within otherwise long-running location series, then exclude gap-affected series when a continuous series is required. section 8.4.1
Customer counts versus segmentation
A customer model contains 35,395 rows for 27,394 unique customers, while roughly 26,000 purchase-data customer IDs have no matching customer details. It can still support a customer count, but its uneven detail makes customer segmentation unsuitable without qualification. Communicate that limitation with the result. section 3.4.4
Completeness checks belong alongside granularity and coverage checks in time-series work. They help determine whether series have comparable amounts of data, whether gaps distort comparisons, and which data can be reshaped, visualized, analyzed, or forecast with defensible assumptions. section 9.3.1
