A black-box model produces predictions or outlier scores whose specific decision process is not readily understandable from the inputs and outputs. The algorithms may be known, while the reasons for individual results remain unclear. section 13.1
Black-box behavior creates practical questions about robustness and fairness. A model may reflect gender or racial bias, use unrepresentative training data, or be underfit or overfit, making future behavior difficult to assess and correct. section 13.1
Interpretability and explainability address the problem differently:
- Prefer an interpretable detector when it provides sufficient accuracy, speed, and understanding. section 13.1.1
- Use post hoc explanations when a more accurate black-box detector is necessary; these explanations approximate why the model assigned a particular score. section 13.1.1
- Distinguish global explanations of overall model behavior from local explanations of individual predictions or outlier scores. section 13.1.1
Outlier detection
Isolation Forest, k-nearest neighbors, Radius, one-class support vector machines, principal component analysis, and related detectors can assign outlier scores without making their specific decisions readily interpretable. A two-dimensional plot may offer useful intuition, but it cannot fully explain outliers in three or more dimensions. section 13.1.1
