liveBook · Manning

Data cleaning

As explained in Grokking Machine Learning

Data cleaning is the process of detecting, correcting, removing, or imputing invalid, inconsistent, or missing data.

Data cleaning prepares messy data for model training. A practical first step is checking which columns contain missing values, then deciding how to handle them. section 13.0 section 13.2

raw_data.isna().sum()
Output:
PassengerId    0
Survived       0
Pclass         0
Name           0
Sex            0
Age          177
SibSp          0
Parch          0
Ticket         0
Fare           0
Cabin        687
Embarked       2

The missing-value counts help guide decisions about which columns to drop or fill. section 13.2

For the Titanic dataset, the missing-value counts inform these treatments:

  • Drop Cabin, which is missing 687 values. section 13.2
  • Fill missing Age values with the column median; 177 values are missing. section 13.2
  • Fill the two missing Embarked values with 'U'. section 13.2

These choices depend on the dataset and the problem being solved. After cleaning, the data can be saved for later model training. section 13.0 section 13.2

How Grokking Machine Learning covers it

How deep it goes
Demonstrates missing-value counts, column removal, median imputation, and filling a categorical value.
Examples
Uses Titanic passenger data, with missing Cabin, Age, and Embarked values, as a cleanup example.
What it assumes
Requires knowledge of the data and problem; uses DataFrames to organize the dataset being cleaned.
What you can do after
Apply it via Pandas to count missing values, drop a column, fill gaps, and save cleaned data as a CSV.
How it connects
Connects cleanup to DataFrames, missing values, Pandas, and model training.
Cover of Grokking Machine Learning
liveBookmanning.com
Read Grokking Machine Learning on liveBook Get the book at manning.com

Also explained in

Compare how 14 books explain data cleaning →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Grokking Machine Learning. The section links point to the parts of the book it draws on.