A surrogate model, also called a proxy or mimic model, is an interpretable model trained to approximate a harder-to-explain model. For outlier detection, it learns to predict a detector’s scores or labels from the original record features. Its rules describe the detector’s estimated behavior, not necessarily every reason a record is unusual. section 13.2.2 section 15.2.2
Choose a form that remains readable while reproducing the detector well enough to be useful:
- A small decision tree exposes feature thresholds and paths; a large tree may be too difficult to interpret. section 13.2.2
- A short rule set or rule list describes groups of records through a limited number of conditions. section 13.2.2
- A restricted linear or logistic regression can provide a compact alternative, though too many terms undermine readability. section 13.2.2
Mimicking an Isolation Forest
Fit an Isolation Forest to abalone records, collect its scores, then train a decision-tree regressor on the original features with those scores as targets. Cross-validate both an unrestricted tree and one limited to 10 leaf nodes; compare the limited tree’s predicted scores with the forest’s scores. The leaf limit makes the tree easier to read but permits only 10 distinct predictions, reducing fidelity. Alternatively, label scores below −0.60 as outliers, cross-validate a decision-tree classifier using macro F1, and inspect its exported thresholds. Paths involving whole weight, shucked weight, and diameter show how combinations of features can lead to different predicted labels. section 13.2.2
Listing 13.4 Creating an IF and a decision tree to mimic its behavior
from sklearn.tree import DecisionTreeRegressor from sklearn.model_selection import cross_validate import matplotlib.pyplot as plt import seaborn as sns np.random.seed(0) regr = DecisionTreeRegressor(random_state=0) #1 cv_results = cross_validate(regr, df[orig_features], df['IF Score'], cv=3) print(cv_results) regr = DecisionTreeRegressor(max_leaf_nodes=10) #2 cv_results = cross_validate(regr, df[orig_features], df['IF Score'], cv=3) print(cv_results) regr = DecisionTreeRegressor(max_leaf_nodes=10) #3 regr.fit(df[orig_features], df['IF Score']) df['DT Prediction'] = regr.predict(df[orig_features]) sns.scatterplot(data=df, x='IF Score', y='DT Prediction') #4 plt.show()
Python workflow for cross-validating an unrestricted and a 10-leaf regressor, then plotting proxy predictions against Isolation Forest scores. section 13.2.2
Listing 13.5 Using a classifier to predict a binary form of the detector score
from sklearn.tree import DecisionTreeClassifier
from sklearn import tree
clf = DecisionTreeClassifier(random_state=0)
df['IF Binary'] = df['IF Score'] < -0.60 #1
cv_results = cross_validate(clf, df[orig_features], df['IF Binary'], cv=3,
scoring='f1_macro') #2
print(cv_results)
clf = DecisionTreeClassifier(random_state=0, max_leaf_nodes=10) #3
cv_results = cross_validate(clf, df[orig_features], df['IF Binary'], cv=3,
scoring='f1_macro')
print(cv_results)
clf = DecisionTreeClassifier(random_state=0, max_leaf_nodes=10) #4
clf.fit(df[orig_features], df['IF Binary'])
df['DT Binary Prediction'] = clf.predict(df[orig_features])
print(tree.export_text(clf, feature_names=orig_features))
Binary-label alternative using a score cutoff, macro-F1 cross-validation, a 10-leaf classifier, and exported tree rules. section 13.2.2
Interpret a surrogate conditionally: a simple model is useful only where its agreement with the detector is adequate. Leaf-by-leaf label counts can reveal which paths mostly capture flagged records and whether different leaves suggest different outlier types. A tree path can also suggest feature changes that would lead to an inlier leaf, but any such counterfactual must be checked against the actual detector and judged for plausibility. Feature importances may identify relevant features without showing their interactions; a surrogate can help with those interactions, but even several explanation methods need not give a complete or fully reliable picture. An interpretable detector used alongside the original model answers a different question: why a record appears anomalous, which may differ from why the original detector flagged it. section 13.2.2 section 13.2.4 section 13.2.5 section 15.2.2
