Data wrangling transforms raw data into a clean, consistent, analysis-ready format. It can reshape tables, filter observations, select variables, convert types, correct inconsistencies, handle missing values, and join sources. The purpose is to make data accurate and usable before analysis or modeling. section 6.2.2 section 6.2.1
A practical wrangling workflow includes:
- Filter records with Boolean criteria, such as retaining fourth-down plays whose type is Run or Pass. section 6.2.2
- Inspect null counts and replace semantically appropriate missing values, such as treating missing YARDS_GAINED as zero for plays with no yards gained. section 6.2.2
- Recode categorical variables into numeric values and derive variables such as CONVERT from YARDS_GAINED and TO_GO. section 6.2.2
- Validate row counts, non-null counts, data types, and visible records before fitting a model or applying a statistical test. section 6.2.2 section 12.4.1
Cleaning leading digits in street addresses
Load only ADDRESS_NO, extract each value's first character, inspect unique values, remove nonnumeric prefixes such as A, B, and T, and convert the remaining first_digit values to integers before evaluating their distribution. section 12.4.1
>>> nfl_pbp = nfl_pbp[(nfl_pbp['DOWN'] == 4) & \ >>> (nfl_pbp['PLAY_TYPE'].isin(['Pass', 'Run']))]
See the Boolean pandas expression that retains only fourth-down Run and Pass plays. section 6.2.2
