Data grouping partitions rows that share key values, producing counts or summary tables for later analysis. In pandas, groupby() supports duplicate checks and entity-level aggregation. section 12.3 section 12.6.1
Choose grouping keys and time windows to match the question:
- Group by date, hour, purchaser, and product to identify repeated transactions; add supplier when the initial result needs closer investigation. section 12.3
- Group by an entity such as Product ID, then calculate means, sums, minima, maxima, or counts before applying outlier tests. section 12.6.1
- Treat time scale as an analytical choice: minute-level checks may be more suspicious, while daily or weekly grouping answers a different frequency question. section 12.3
Transaction duplicate screening
Grouping purchases by date, hour, staff, and product found 74 pairs of matching purchases and no groups of three or more. Adding supplier reduced the potentially actionable cases to five, which could then be investigated for recurring purchaser, product, supplier, or time patterns. section 12.3
Listing 12.2 Testing for duplicates
duplicates_test_df = (pd.DataFrame(
purchases_df.groupby(
['Date', 'Hour', 'Staff ID', 'Product ID']
).size(),
columns=['Count']
)).reset_index()
duplicates_test_df['Count'].value_counts()
Shows the pandas pattern for grouping transaction keys, counting rows, resetting the index, and inspecting group-size frequencies. section 12.3
purchases_df.groupby(['Product ID']).agg({
'Count': ['mean', 'sum'],
'Unit Cost': ['mean', 'sum'],
'Total Cost': ['mean', 'sum'],
'Hour': ['mean', 'min', 'max'],
'Purchase ID': ['count']})
Shows entity-level aggregation with multiple statistics across transaction counts, costs, hours, and purchase identifiers. section 12.6.1
