liveBook · Manning

Multi-column index

As explained in Pandas Workout: 200 exercises to make you a stronger data analyst

A pandas MultiIndex is a hierarchical index formed from multiple levels of labels.

A pandas MultiIndex is a hierarchical index with multiple label levels. Use it when one row label is not enough—for example, sales broken down by year and month, or results broken down by year and state. Each level provides another route into the same data, so queries can start high in the hierarchy or target an exact combination. section 4.0 section 4.4

Create a row MultiIndex by passing several columns to set_index, or create it while reading a CSV with index_col. A two-level index such as year and month lets year select all its months, while a tuple such as (2018, 'Jun') selects one specific row. Lists can select several labels at a level; use slice(None) when a level should accept everything. section 4.4 section 4.6

Analyze SAT scores by year and state

Make Year and State.Code the index, then express the requested levels together inside a tuple. This selects four states in 2010, retrieves one data column, and calculates its mean: df.loc[(2010, ['NY', 'NJ', 'MA', 'IL']), 'Total.Math'].mean() For several years and states, place a list at each corresponding index level. The key reasoning step is separating index selectors from the final column selector. section 4.6 section 4.5

Deeper indexes work the same way, but level order matters. An Olympics index ordered as Year, Season, Sport, and Event can select a year range and Summer while leaving later levels unrestricted. Sort the index early: multi-index operations can work better on sorted data and may otherwise produce a PerformanceWarning. section 4.10 section 4.9

Slice selected hierarchy levels

Use a tuple in .loc, with one selector per level. Here, years 1936–2000 and Summer are constrained; Sport and Event are implicitly left open: df.loc[(slice(1936, 2000), 'Summer'), 'Age'].mean() To target Archery across all years, use slice(None) for Year. Filter missing medals before counting winning teams. section 4.10 section 4.11

When a query cares about a named inner level rather than its position, xs can be shorter than a .loc tuple. IndexSlice offers ordinary colon slicing inside square brackets, avoiding repeated slice(None) calls. Named levels are encouraged because they are easier to read than integer level positions. section 4.11 section 4.11

MultiIndexes can also label columns. Split flat survey names such as other.lang.Java at their final period, convert the resulting pairs with pd.MultiIndex.from_tuples, assign them to df.columns, and sort the columns. Selecting py_df['other.lang'] then exposes the related second-level language columns as a group. section 8.1 section 8.2

Figure 4.4 Graphical depiction of our data frame with four columns in its multi-index

Inspect the four-level Olympic layout before writing selectors; it makes the level order behind each tuple visible. section 4.10

How Pandas Workout: 200 exercises to make you a stronger data analyst covers it

How deep it goes
Works through creation, sorting, tuple slicing, xs, IndexSlice, and hierarchical column labels without describing pandas internals.
Examples
Uses sales by year/month, SAT scores by year/state, Olympic events, and hierarchical Python survey columns.
What it assumes
Builds on DataFrame indexes and loc selection while extending them from one label level to several.
What you can do after
Apply it via pandas to build hierarchical row or column labels and compute aggregates from selected slices.
How it connects
Connects MultiIndex work to DataFrame indexes, loc, set_index, read_csv, pivot tables, xs, and IndexSlice.

This explainer was generated by Manning's knowledge graph from the text of Pandas Workout: 200 exercises to make you a stronger data analyst. The section links point to the parts of the book it draws on.