Data cleaning turns inconsistent, incomplete, or ambiguous records into data suitable for analysis. A practical workflow inspects quality first, preserves raw values, and adds cleaned fields for comparison. section 2.2.2
A useful cleaning workflow includes:
- Check shape and missing values before transforming records; decide whether incomplete rows should be removed or retained. section 2.2.2
- Standardize representations such as letter casing, punctuation, and unwanted markers. section 2.2.2
- Explore structural variation instead of assuming every record has the same number or position of fields. section 2.2.2
- Use an external reference list to classify records, then inspect unmatched cases and add justified exceptions. section 2.2.2
- Keep an explicit fallback category such as OTHER when classification remains unresolved, and account for it in later analysis. section 2.2.2
Distinguishing London addresses
Searching for LONDON can incorrectly include addresses such as London Road. Uppercasing a separate address_clean column and matching the address-line pattern LONDON, reduces this ambiguity; comparing both counts reveals the practical effect of the rule. section 2.2.2
customers["address_clean"] = customers["address"].str.upper()
Inspect the executable transformation that creates a consistently cased address column while preserving the original address. section 2.2.2
