Data aggregation combines raw records into usable summaries for downstream forecasting and feature calculation. In a retail pipeline, transactions are aggregated daily before other data sources are joined. section 6.7.1
A practical aggregation workflow should:
- Check transactions, stock, and promotion data for missing or duplicated values before aggregation. section 6.7.1
- Add each newly aggregated daily partition to the transaction-aggregate table. section 6.7.1
- Optionally rewrite the previous two or three days to repair duplicates, incomplete data, or corruptions. section 6.7.1
- Join internal and external data by date, product ID, or store ID, then calculate features. section 6.7.1
Retail forecasting pipeline
A retailer aggregates transaction history daily, joins it with stock, promotions, weather, product, store, and competitor-price data, and derives forecasting features. Gaps in competitor coverage require explicit handling. section 6.7.1
