liveBook · Manning

CategoricalDtype

As explained in Pandas Workout: 200 exercises to make you a stronger data analyst

CategoricalDtype is a pandas type specification that defines the categories and ordering for categorical data.

Treat CategoricalDtype as a predefined vocabulary for a pandas column: declare every permitted value before converting the data. section 12.0

Allow a future category

A plain categorical Series rejects a later assignment such as s.loc[7] = 'd'. Define the complete category set first, then convert with it: abcd_category = pd.CategoricalDtype(['a', 'b', 'c', 'd']); s = Series(['a', 'b', 'c', 'a', 'b', 'c', 'c', 'c']).astype(abcd_category); s.loc[7] = 'd'. The assignment succeeds because 'd' was declared in advance. section 12.0

Apply a schema to a column

For Olympic medal data, define medals_category = pd.CategoricalDtype(['Gold', 'Bronze', 'Silver']) and assign df['Medal'] = df['Medal'].astype(medals_category). The same pattern supplies a controlled categorical type to an existing column. section 12.0

Table 12.1 What you need to know (view table figure)

Concept

What is it?

Example

To learn more

df.info

Gets information about a data frame, including its memory usage

df.info()

http://mng.bz/D4eE

df.memory_usage

Gets information about a data frame’s memory usage

df.memory_usage (deep=True)

http://mng.bz/lWjy

Categorical data

Pandas documentation for categorical data

df['a'].astype ('categorical')

http://mng.bz/BmaJ

df.to_feather

Writes a data frame to feather format

df.to_feather ('mydata.feather')

http://mng.bz/d1pQ

pd.read_feather

Creates a data frame based on a feather-formatted file on disk

df = pd.read_feather ('mydata.feather')

http://mng.bz/rWlX

pd.read_csv

Returns a new data frame based on CSV input

df = df.read_csv ('myfile.csv')

http://mng.bz/V1r5

pd.read_json

Returns a new data frame based on JSON input

df = df.read_json ('myfile.json')

http://mng.bz/x4qB

time.perf_counter

Gets the number of seconds (useful for timing programs)

time.perf_counter()

http://mng.bz/AonW

df.query

Writes an SQL-like query

df.query('v > 300')

http://mng.bz/ZqBZ

df.eval

Performs actions and queries on a data frame

df.eval('v + 300')

http://mng.bz/Rx9P

pd.eval

Performs a variety of pandas actions in an evaluated string

pd.eval('df.v > 300')

http://mng.bz/2D9X

timeit

Python module for benchmarking code speed, and a Jupyter “magic command” for invoking it

%timeit 3+2

http://mng.bz/1qKg

isin

Checks whether a value is in a Python sequence

df['a'].isin ([10, 20, 30])

http://mng.bz/Pz0P

pd.CategoricalDtype

Returns a new categorical dtype

pd.CategoricalDtype (['a', 'b', 'c', 'd'])

http://mng.bz/Jgev

Use the categorical-data entry to connect explicit type conversion with related pandas memory and data-file operations. section 12.0

How Pandas Workout: 200 exercises to make you a stronger data analyst covers it

How deep it goes
Explains construction, conversion, and prevention of invalid assignments through a small executable example.
Examples
Uses a Series that later receives 'd' and an Olympic Medal column with Gold, Bronze, and Silver categories.
What it assumes
Assumes familiarity with pandas Series, DataFrame columns, and astype conversion.
What you can do after
Apply it via pandas to permit planned future category values during categorical conversion.
How it connects
Connects categorical data with Series.astype and DataFrame columns.
Cover of Pandas Workout: 200 exercises to make you a stronger data analyst
liveBookmanning.com
Read Pandas Workout: 200 exercises to make you a stronger data analyst on liveBook Get the book at manning.com

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Pandas Workout: 200 exercises to make you a stronger data analyst. The section links point to the parts of the book it draws on.