A missing value marks an observation or field that was never collected, became unavailable, or cannot be used as recorded. In pandas, missing data is commonly represented by NaN rather than 0, because zero can be a meaningful measurement. NaN is a floating-point value that cannot become an integer and is not equal to itself. section 2.20
Handle missing values by matching the cleaning rule to the analytical question:
- Remove rows containing any NaN with dropna(); this is simple but can discard every row when each row has one missing field. section 5.2
- Require completeness only in decision-critical columns by passing their names through dropna(subset=[...]). section 5.1 section 5.2
- Allow limited gaps with thresh, which keeps rows containing at least a specified number of non-null values. section 5.4
Parking-ticket cleanup
Load selected columns from the NYC parking-violations CSV, count rows with len(df.index), and compare several policies. First, remove every incomplete row with df.dropna(). Then retain rows only when Plate ID, Registration State, Vehicle Make, and Street Name are present. Finally, relax the rule by omitting Vehicle Make. Subtract the retained-row count from the original count and multiply by $100 to estimate the potential cost of dismissible tickets. Test a further variation with thresh=3, and convert BLANKPLATE into NaN before rerunning the analysis. section 5.1 section 5.2 section 5.4
df = df.dropna().copy()
Use df.dropna().copy() when you need a separate, fully non-null data frame for further analysis. section 2.20
