A classifier predicts which predefined category an observation belongs to. It learns from labeled examples: an image labeled cat, dog, or hamster supplies a category to predict for a new image. Unlike a regressor, which predicts a numeric value, a classifier predicts a class. In outlier work, the classes might be normal and a known failure type, or categories assigned to records already flagged for investigation. section 1.2 section 15.3.2
First decide whether the goal is to recognize a specific, known issue or to find anything statistically unusual. A classifier can learn a known issue when representative labeled examples exist. It is less suited to discovering unforeseen anomalies: rare events may have too few training examples, and new outliers may differ from every category it has learned. Rules may be simpler and more interpretable for a clear pattern; an outlier detector is generally the better starting point for unknown statistical outliers. These approaches can also be combined. section 4.3 section 4.3.3
Recognizing a known failure
Suppose machinery logs cover several past failures and many periods of normal operation. If the warning pattern is too complicated to express as a useful rule, label the two kinds of periods and train a classifier to distinguish them. Check that the scarce failure examples adequately represent the failures to be found later; a large dataset does not by itself guarantee enough minority-class examples. section 4.3.3
A classifier can also support an outlier-detection system without serving as its sole detector:
- Distinguish real from generated records. A real-versus-fake detector treats real records predicted to look synthetic as candidates for investigation. Its result depends on both the synthetic-data generator and the classifier. section 3.2.5 section 11.5
- Explain another detector approximately. A decision-tree classifier can learn binary labels obtained by thresholding detector scores; its readable splits describe the proxy’s decisions, not necessarily every decision of the original detector. section 13.2.2
- Combine detectors. Given ground-truth labels, a stacked ensemble can train a classifier on several detectors’ scores and use its positive-class probability as a final outlier score. Unrepresentative labels can mislead it. section 14.6.5
- Sort flagged records. After investigators label examples of useful categories, a classifier can route later flagged records for grouped investigation or filtering. Classification follows detection here; it does not establish that a record has no other anomaly. section 15.3 section 15.3.3
Figure 15.8 Left pane: dataset with known outliers. Inliers are drawn as dots and outliers as stars. Conceivably, we can create a classifier to separate these, but as the outliers are distributed throughout the space, it is not clear where to draw the decision boundary. This is much more of a problem in higher dimensions. Right pane: where there are many examples each of a small number of specific outliers, it is more feasible to classify these.
The contrasting decision boundaries show why scattered, previously unseen outliers resist a general normal-versus-abnormal classifier, while repeated examples of a few specific outlier types are more feasible to classify. section 15.3.2
When sorting flagged records, compare a record’s detector score with scores typical of its assigned category. An unusually high score may indicate an additional, unknown pattern—or an extreme instance of the known one. Keep human review and consistent labels in the loop; accumulated labels can improve later classifiers and help test changes to the detection system. section 15.3.3 section 15.6
