How deeply does The Well-Grounded Data Analyst cover transform?

Demonstrates grouped `sum` transformation in a worked pandas calculation and verifies its output against raw records.

Transform

As explained in The Well-Grounded Data Analyst

The pandas transform method applies a function to grouped data and returns results aligned with the original data shape.

Use pandas `transform` when a grouped calculation must remain beside the rows that produced it. It supports window-function-like analysis without creating a separate summary table and joining it back. section 9.2.1

Hourly share of daily cycling

After aggregating cycling counts to one row per location, date, and hour, group by location and date and transform `Pedal_cycles` with `sum`. The resulting daily total is repeated on each hourly row, so dividing the hourly count by that total yields `hourly_pct`. Check a location's raw records to confirm both the total and percentages before using them. section 9.2.1

cycling_daily_hourly['TotalDailyCount'] = (
    cycling_daily_hourly
    .groupby(['Count_point_id', 'Count_date'])
    ['Pedal_cycles']
    .transform('sum')     #1
)

cycling_daily_hourly['hourly_pct'] = (
    cycling_daily_hourly['Pedal_cycles']
    / cycling_daily_hourly['TotalDailyCount']
)

cycling_daily_hourly

This listing appends each group's daily total, then calculates the hourly proportion from values aligned to the hourly rows. section 9.2.1

How The Well-Grounded Data Analyst covers it

liveBook manning.com

Read The Well-Grounded Data Analyst on liveBook Get the book at manning.com

Related concepts

grouped dataframeused by · A grouped dataframe is a pandas object that partitions dataframe rows into groups for separate aggregation or transformation.
Seriesproduces · A Series is pandas’ one-dimensional labeled data structure that associates values with an index.
broadcastingused by · Broadcasting is the distribution or implicit replication of data from one source across multiple recipients or compatible positions for communication or computation.
lambdaused by · A lambda is a language construct that defines an unnamed function or callable, often as a value.
window functionalternative to · A window function is an SQL operation that computes a value over related rows while preserving each individual row in the result.

This explainer was generated by Manning's knowledge graph from the text of The Well-Grounded Data Analyst. The section links point to the parts of the book it draws on.