The pandas merge function combines related tables by matching shared keys, such as product_id. It supports a staged alternative to one large SQL join, keeping intermediate data ready for further transformations or machine-learning preparation. section 2.3.1
Combining product tables
Suppose separate tables contain product names and prices, descriptions and categories, and manufacturer details. Merging them on product_id produces one analysis-ready product table. section 2.3.1
Listing 2.4 Merging datasets in pandas
import pandas as pd
df1 = pd.DataFrame({'product_id': [1, 2, 3, 4],
'product_name': ['Product A',
'Product B',
'Product C',
'Product D'],
'price': [10.99, 20.99, 15.99, 8.99]}) #1
df2 = pd.DataFrame({'product_id': [1, 2, 3, 4],
'product_description': ['A great product',
'A high-quality product',
'A reliable product',
'An affordable product'],
'category': ['Category A', 'Category B',
'Category C', 'Category D']}) #2
df3 = pd.DataFrame({'product_id': [1, 2, 3, 4],
'manufacturer': ['Manufacturer A', 'Manufacturer B',
'Manufacturer C', 'Manufacturer D'],
'weight': [1.5, 2.0, 1.8, 1.2]}) #3
merged_df = pd.merge(df1, df2, on='product_id') #4
merged_df = pd.merge(merged_df, df3, on='product_id') #5
print(merged_df)
Inspect the pandas listing to see three DataFrames built around product_id and merged progressively. section 2.3.1
Choose merge for practical, inspectable table preparation, while recognizing its limits:
- Preserve control over each merge stage and intermediate result. section 2.3.1
- Move to tools such as Dask when data exceeds available memory or pandas becomes too slow. section 2.3.1
