Association rule mining describes relationships between categorical values with rules of the form X → Y. A frequent item set may span one or more columns, such as area code 416 being associated with Toronto. section 7.10
Evaluate records in two complementary ways:
- Count how many association rules hold for each row; fewer satisfied rules indicate greater outlierness. section 7.10
- Find exceptions where an antecedent is present but its consequent is absent, such as area code 416 without Toronto. section 7.10
- Combine exception testing with another detector when rows containing few or no frequent item sets also matter. section 7.10
The exception-based approach is interpretable: each flagged row can be linked to rules whose left-hand side holds while the right-hand side does not. It therefore identifies a different anomaly type from methods based mainly on sparse frequent-itemset membership. section 7.10
Listing 7.12 Creating frequent item sets
from mlxtend.frequent_patterns import apriori
import pandas as pd
from sklearn.datasets import fetch_openml
import matplotlib.pyplot as seaborn
data = fetch_openml('SpeedDating', version=1, parser='auto') #1
data_df = pd.DataFrame(data.data, columns=data.feature_names)
data_df = data_df[['d_pref_o_attractive', 'd_pref_o_sincere',
'd_pref_o_intelligence', 'd_pref_o_funny',
'd_pref_o_ambitious', 'd_pref_o_shared_interests']] #2
data_df = pd.get_dummies(data_df) #3
frequent_itemsets = apriori(data_df, min_support=0.3, use_colnames=True) #4
Create one-hot categorical columns, then mine frequent item sets with Apriori and a chosen minimum support. section 7.10
Listing 7.13 Creating and testing association rules
from mlxtend.frequent_patterns import association_rules
assoc_rules = association_rules(frequent_itemsets, metric="confidence",
min_threshold=0.7)
Generate rules from the frequent item sets by selecting confidence and a minimum threshold. section 7.10
SpeedDating categorical preferences
After encoding preference ranges, rules can associate high attractiveness preferences with low ambition preferences or low shared-interests preferences. A row that satisfies the antecedent but not the consequent becomes a rule exception.
