liveBook · Manning

Set_index

As explained in Pandas Workout: 200 exercises to make you a stronger data analyst

The pandas method that replaces an object's index with values from one or more specified columns or arrays.

`set_index` replaces a DataFrame’s current index with values from one column or from several columns. This is useful when the values you repeatedly ask about already live in a column: make that column the row label, then retrieve matching rows with `loc`. Think of it as moving a useful field from the body of a table onto the tabs used to organize it. section 4.2 section 4.4

Find tickets for a particular date

A parking-ticket table has an `Issue Date` column. If date-based queries are the task, assign the returned DataFrame back to `df` after setting that column as the index. Then `df.loc['01/02/2020 12:00:00 AM']` selects tickets from that date. Add a second `loc` argument to select a column, such as `Vehicle Make`, and continue with `value_counts()` to find common makes. section 4.2

df = df.set_index('Issue Date')

This one-line transformation makes `Issue Date` the row index; subsequent label lookups use issue dates rather than the former default index. section 4.2

The method returns a new DataFrame. Assigning that result is the preferred pattern: it preserves method chaining and avoids `inplace=True`, which returns `None` while modifying the invoked DataFrame. Once a column becomes the index, it is no longer available as an ordinary named column to many operations, although some methods such as `groupby` can work with the index by its original name. section 4.2

To choose a different index later, first restore the existing index with `reset_index()`, then call `set_index` again. For example, a date-indexed ticket table can become color-indexed with `df.reset_index().set_index('Vehicle Color')`. That shape supports selecting several colors with `df.loc[['BLUE', 'RED']]` before narrowing to a column and summarizing results. section 4.2

Create a hierarchy for sales data

Pass a list of columns to create a multi-index: `df.set_index(['year', 'month'])`. Put the more general level first, so year is outside month; similarly, geographical labels commonly run from region to country to city. With year and month indexed, `df.loc[2018]` retrieves all 2018 rows, while `df.loc[(2018, 'Jun')]` targets the specific year-month combination. The row selector is still the first `loc` argument, and a column selector remains the second. section 4.4 section 4.4

df = df.set_index(['year', 'month'])

This listing shows the compact multi-index form: supplying `year` and `month` together makes both available for hierarchical row selection. section 4.4

A multi-index supports analytical questions across combinations of labels. A SAT-score table indexed by `Year` and `State.Code` can sum 2005 test-takers by selecting the year and `Total.Test-takers`, or calculate mean 2010 math scores for selected state codes with a tuple containing the year and a list of states. The same index can be created during CSV loading with `read_csv(..., index_col=['Year', 'State.Code'])`. section 4.6 section 4.6

Align an index for a join

`set_index` also prepares tables for joining when matching labels need to align. An OECD-country table uses country names as its index, and setting `Country` as the index on survey data allows a join before building a salary-by-country-and-education pivot table. Here the index change is temporary within a method chain and serves the later join. section 8.2

(
    oecd_df
    .join(so_df
          .set_index('Country'))
    .pivot_table(index='Country',
                 columns='EdLevel',
                 values='ConvertedCompYearly')
)

The chained example uses `so_df.set_index('Country')` immediately before `join`, making the matching country labels explicit before the pivot table is created. section 8.2

How Pandas Workout: 200 exercises to make you a stronger data analyst covers it

How deep it goes
Works through returned-object behavior, index replacement, multi-index ordering, and tuple-based `loc` selections; defers broader index internals.
Examples
Uses New York City parking tickets, monthly product sales, SAT scores, and country-based salary analysis.
What it assumes
Assumes familiarity with DataFrames, columns, boolean masks, and `loc` while extending them to index-based selection.
What you can do after
Apply it via pandas to reshape DataFrames for label-based retrieval, hierarchical analysis, and aligned joins.
How it connects
Connects `set_index` with `loc`, `reset_index`, `groupby`, `read_csv`, `join`, and `pivot_table`.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Pandas Workout: 200 exercises to make you a stronger data analyst. The section links point to the parts of the book it draws on.