A label is a value associated with a data item that identifies a target category or numeric outcome. In supervised learning, labels provide the values a model learns to predict: animal-image categories support classification, while a numeric count of animals supports regression. Outlier detection often starts without such labels, so a detector’s output should not be mistaken for a known answer. section 1.2
In outlier detection, distinguish three uses of labels:
- A detector may return a binary inlier/outlier label. A numeric outlier score instead ranks how unusual records appear; converting scores to labels requires a threshold that may have no natural cutoff. section 3.3.4
- A ground-truth label supplies a reference for evaluation. Synthetic anomalous records can provide such labels even when the original records remain unlabeled, but the distinction is an assumption about the generated data. section 8.5.2
- A hand-applied label describes whether a flagged record matters to a project and, preferably, what kind of issue it represents. Statistical unusualness alone does not establish practical relevance. section 15.1
Keep scores when the task is to review a fixed number of records: the top 50 can be selected from a ranking, whereas binary flags might identify more or fewer than 50. With evaluation labels, AUROC measures how well outlier scores rank presumed outliers above presumed inliers; precision at k addresses how many of the top k reviewed records are outliers. Rank well before choosing thresholds for label-based measures such as precision, recall, and F1. section 3.3.4 section 8.5.2
For investigated records, define recurring categories clearly and apply them consistently. A record may need multiple labels; a statistically unusual but uninteresting record can also receive a specific label. Categories can guide investigations, tracking, and detector retuning. Rules or classifiers may assign some labels automatically, but novel cases may still require hand-labeling or confirmation. section 15.1.1 section 15.2
A label need not belong to one unusual record
In simulated purchases, a rogue purchaser has a known pattern of escalating and typically larger purchases. Individual transactions need not be striking: the anomalous behavior appears across the purchaser’s records. A useful evaluation reference must therefore reflect the behavior being sought, and simulated test data must remain plausible. section 12.1
Figure 3.8 The distribution of outlier scores using FPOF on the SpeedDating dataset and KDE on the wall-robot-navigation dataset. In some cases, such as the KDE scores from the robot navigation dataset, we can identify a reasonable cutoff to separate inliers from outliers, while with the FPOF scores from SpeedDating, there is no clear cutoff.
The two score distributions illustrate why one dataset may permit a reasonable inlier/outlier cutoff while another does not. section 3.3.4
