liveBook · Manning

Annual aggregation

As explained in The Well-Grounded Data Analyst

Annual aggregation is the process of summarizing observations into values representing individual calendar years.

Annual aggregation summarizes observations into one value for each calendar year. For cycling traffic, summing hourly pedal-cycle counts by location ID and year produces a compact series for comparing long-term change while removing hourly noise. section 9.2.1

Annual cycling counts by location

Group traffic records by Count_point_id and Year, sum Pedal_cycles, and reset the index. The result has one row per location-year, ready for trend comparisons or later calculations of absolute and percentage change. section 9.2.1

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
traffic = pd.read_parquet("./data/time_series.parquet.gz")

cycling = (
    traffic
    .groupby(["Count_point_id", "Year"])
    ["Pedal_cycles"]
    .sum()
    .reset_index()
)

cycling.head()

This pandas workflow reads the prepared traffic data and creates annual pedal-cycle totals for every location. section 9.2.1

Inspect the aggregated distribution before selecting locations: many locations may have little or no cycling traffic, so histograms and quartiles help choose a defensible cutoff. section 9.2.1

How The Well-Grounded Data Analyst covers it

How deep it goes
Demonstrates pandas grouping and summation, then uses annual values for distribution and change analysis.
Examples
Uses hourly pedal-cycle counts grouped by location ID and calendar year.
What it assumes
Assumes cleaned, complete traffic data with location, year, and pedal-cycle fields.
What you can do after
Apply it via pandas to create one annual total per location from hourly traffic records.
How it connects
Connects time series data with pandas groupby, distributions, histograms, quartiles, and percentage change.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of The Well-Grounded Data Analyst. The section links point to the parts of the book it draws on.