An expected outlier is rare or extreme, but belongs to a kind of record that can reasonably occur. This can mean an extreme value that is plausible under a distribution, or a type of unusual record already seen or anticipated. It does not mean the record is common or unimportant. section 2.1.1 section 4.2.3
Plausible versus impossible height
A recorded height over 7 feet may be an outlier yet plausible in a sufficiently large dataset. A height of 70 feet instead points to a data error. Likewise, an autonomous vehicle might rarely encounter another vehicle travelling at extremely high speed: unusual, but imaginable in advance. section 2.1.1 section 4.2.3
Expectedness depends on both the data and the question. For roughly Gaussian data, probabilities associated with z-scores can help choose a context-appropriate threshold; do not assume every dataset has that distribution. A statistical outlier can be either anticipated or unanticipated, while a search for a specific anomaly usually targets a known type. Rules, classifiers, and outlier detectors are alternative or complementary ways to look for these records. section 2.1.1 section 4.2.3 section 4.3
To identify relevant outliers already present in a dataset:
- Run several detectors with varied preprocessing and parameters; collect records receiving high scores from at least one detector and remove duplicates. section 14.4.1
- Inspect the candidates, and potentially records just below the cutoffs, to decide which matter for the project. Treat the resulting set as known outliers in the current data, not proof that every relevant outlier was found. section 14.4.1
