liveBook · Manning

Normalization

As explained in Pandas Workout: 200 exercises to make you a stronger data analyst

Normalization is the organization of data into separate related structures so that each piece of information is stored only once.

Normalization organizes information so each fact appears once, either by standardizing inconsistent values or separating related data into structures that can be joined when needed. section 5.13 section 6.8

Standardizing vehicle colors

A parking-violations dataset may record WHITE as WHITE, WT, WT., and WHT. Count distinct values, inspect the most common colors, build a dictionary mapping alternate inputs to canonical names, replace them, and inspect the remaining inconsistencies and errors. section 5.13

Separating products and sales

Keep product details in products_df and transactions in sales_df, set product_id as the index in both, and join the frames. The combined data supports totals by product, income by product, daily income, and product income by day. section 6.8

products_df.join(sales_df)

A concise pandas join makes normalized product and sales data available for subsequent grouping and sorting. section 6.8

How Pandas Workout: 200 exercises to make you a stronger data analyst covers it

How deep it goes
Works through value mapping and indexed joins, but stops short of formal normalization theories.
Examples
Uses inconsistent country and vehicle-color labels, plus separated product and sales tables.
What it assumes
Assumes familiarity with Python data frames, columns, indexes, and basic grouping or sorting operations.
What you can do after
Apply it via pandas by mapping alternate values and joining normalized data frames for analysis.
How it connects
Connects data cleaning, data frames, indexes, joins, grouping, and sorting.

Also explained in

Compare how 21 books explain normalization →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Pandas Workout: 200 exercises to make you a stronger data analyst. The section links point to the parts of the book it draws on.