This chapter covers
- The benefits and tradeoffs in creating ensembles
- Selecting the detectors for ensembles
- Scaling the scores from the detectors
- Combining the scores from each detector for a final score
Often when evaluating outlier detectors, we’re able to identify a number of detectors that appear to work well, though none work perfectly. We may also have cases where we appear to detect the known outliers that we test with well but are not confident we will detect the full range of outliers we may encounter in the future. In most cases, the solution in these situations is to use multiple outlier detectors, combining them into an ensemble. This is a powerful technique, and it’s very common to use ensembles for outlier detection problems.
This is similar to the idea of creating ensembles with prediction problems, where ensembling is well understood to be a very powerful technique. In fact, with tabular data, the strongest models tend to be ensembles, for example XGBoost, LGBM, and CatBoost, which are ensembles of decision trees. In addition, the strongest autoML tools, such as AutoGluon, focus on creating ensembles of models as the most effective means to create highly accurate predictive systems.