liveBook · Manning

ECOD

As explained in Outlier Detection in Python

An outlier detection algorithm that estimates feature-wise tail extremeness from empirical cumulative distributions and aggregates it into an outlier score.

ECOD detects outliers by estimating each feature's tail extremeness from its empirical cumulative distribution function (ECDF), then aggregating feature-wise scores. An ECDF uses the observed data itself: for each value, it records the fraction of observations less than or equal to it. section 6.3

Use ECOD when a fast, deterministic, parameter-free detector and feature-level explanations are useful:

  • Fit it without choosing bins or other tuning parameters, which avoids a major sensitivity of histogram-based detection. section 6.3
  • Interpret high scores through unusually extreme individual feature values, rather than assuming that every flagged row represents a meaningful anomaly. section 6.3 section 15.2.2
  • Recognize its limits: ECOD misses unusual combinations of otherwise ordinary feature values and detects extreme values rather than internal outliers. section 6.3
  • Pair it with a multivariate detector when needed; ECOD explanations can partially clarify outliers identified by Isolation Forest. section 15.2.2

Applying ECOD to tabular data

With PyOD, create an ECOD detector, fit it to the speech dataset, read decision_scores_, and explain the row with the largest score. On a 400-feature example, the most anomalous row has a high fraction of values above one or both cutoff bands. section 6.3

Listing 6.6 Example using an ECDF plot from seaborn
from sklearn.datasets import load_breast_cancer
import matplotlib.pyplot as plt
import seaborn as sns
 
data = load_breast_cancer()['data']
col = 3
 
fig, ax = plt.subplots(nrows=1, ncols=4, figsize=(11, 2))
sns.histplot(data[:, col], ax=ax[0], bins=5)              #1
sns.histplot(data[:, col], ax=ax[1], bins=20)             #2
sns.histplot(data[:, col], ax=ax[2], bins=200)            #3
sns.ecdfplot(data[:, col], ax=ax[3])                      #4
plt.tight_layout()
plt.show()

Compare histograms with different bin counts against an ECDF to see why ECOD avoids bin-selection sensitivity. section 6.3

How Outlier Detection in Python covers it

How deep it goes
Explains ECDF construction, implementation, interpretation, and limitations, but stops short of deriving the scoring formula.
Examples
Demonstrates ECDF plots on breast cancer data and ECOD explanations on speech and KDD Cup datasets.
What it assumes
Requires tabular feature values and teaches empirical cumulative distributions from observed sorted values.
What you can do after
Apply it via PyOD, inspect decision scores, explain flagged rows, and compare its results with Isolation Forest.
How it connects
Connects empirical cumulative distributions with HBOS, COPOD, and Isolation Forest.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Outlier Detection in Python. The section links point to the parts of the book it draws on.