liveBook · Manning

Preaggregation

As explained in Machine Learning System Design

The technique of aggregating lower-level data before applying further aggregations or feature calculations.

Preaggregation speeds feature calculation by aggregating raw data before applying further aggregations. Instead of repeatedly scanning clicks, prices, or revenue, calculate reusable item-day sums and build larger windows from them. section 11.4.2

From daily data to rolling features

A feature pipeline can first produce item-day totals, then combine those totals into 7-, 14-, 30-, or 60-day windows. Updating features after the database processes the previous day’s orders allows yesterday’s or older calculations to be reused rather than recomputed. section 11.4.2

Figure 11.14 A hierarchy of aggregating used in a feature store
figure

The aggregation hierarchy makes the reusable progression from lower-level summaries to larger feature windows visible. section 11.4.2

How Machine Learning System Design covers it

How deep it goes
Explains the multi-step aggregation pattern and reuse rationale, but does not detail implementation algorithms.
Examples
Uses clicks, prices, revenue, item-day sums, order processing, and 7- to 60-day aggregation windows.
What it assumes
Assumes familiarity with raw event data, feature calculation, aggregation windows, and feature stores.
What you can do after
Apply preaggregation to reuse intermediate feature calculations and avoid nearly identical repeated computations.
How it connects
Connects preaggregation with feature stores, batch features, feature versioning, and feature dependency graphs.
Cover of Machine Learning System Design
liveBookmanning.com
Read Machine Learning System Design on liveBook Get the book at manning.com

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Machine Learning System Design. The section links point to the parts of the book it draws on.