A tail probability measures how likely a random variable is to take a value at least as extreme as a chosen boundary in a distribution’s tail. For a value on an empirical cumulative distribution function (ECDF), the probability of observing a larger value is one minus the cumulative probability at that point. section 6.3
Tail probabilities support practical outlier detection in several related ways:
- ECOD estimates tail probabilities separately for each feature, detects extreme values in one or both tails, and aggregates the feature results by multiplication. Its aggregate score represents unusualness rather than a true probability. section 6.3
- COPOD builds an empirical copula from feature ECDFs, estimates how extreme each record is jointly, adjusts for dimensionality, and corrects for skew. section 6.4
- Interpret tail evidence cautiously: symmetric distributions may make both tails relevant, while skewed distributions may make only one tail meaningful. section 6.3
Synthetic two-feature outlier
Generate two normally distributed features centered near 10, replace the final row’s first feature with 15, and fit COPOD. The explanation identifies the first feature as extreme while leaving the second unremarkable. section 6.4
Listing 6.6 Example using an ECDF plot from seaborn
from sklearn.datasets import load_breast_cancer import matplotlib.pyplot as plt import seaborn as sns data = load_breast_cancer()['data'] col = 3 fig, ax = plt.subplots(nrows=1, ncols=4, figsize=(11, 2)) sns.histplot(data[:, col], ax=ax[0], bins=5) #1 sns.histplot(data[:, col], ax=ax[1], bins=20) #2 sns.histplot(data[:, col], ax=ax[2], bins=200) #3 sns.ecdfplot(data[:, col], ax=ax[3]) #4 plt.tight_layout() plt.show()
An ECDF plot makes tail probability visible: steep regions contain many observations, while flat tail regions contain few. section 6.3
