liveBook · Manning

Empirical cumulative distribution function

As explained in Outlier Detection in Python

A function that assigns each value the proportion of observed data points less than or equal to it.

An empirical cumulative distribution function (ECDF) assigns each observed value the proportion of data points less than than or equal to it. Unlike a theoretical cumulative distribution, it uses the actual sample. For sorted values, the function increases as observations accumulate, showing where data are concentrated and where values are relatively rare. section 6.3

Listing 6.6 Example using an ECDF plot from seaborn
from sklearn.datasets import load_breast_cancer
import matplotlib.pyplot as plt
import seaborn as sns
 
data = load_breast_cancer()['data']
col = 3
 
fig, ax = plt.subplots(nrows=1, ncols=4, figsize=(11, 2))
sns.histplot(data[:, col], ax=ax[0], bins=5)              #1
sns.histplot(data[:, col], ax=ax[1], bins=20)             #2
sns.histplot(data[:, col], ax=ax[2], bins=200)            #3
sns.ecdfplot(data[:, col], ax=ax[3])                      #4
plt.tight_layout()
plt.show()

Plot an ECDF beside histograms with different bin counts to see why the ECDF avoids the arbitrary binning choice: steep regions contain many observations, while flatter regions contain few. section 6.3

ECDFs support fast, distribution-free outlier detection when applied feature by feature. ECOD uses them without tuning parameters, making the detector deterministic and scalable, but the independent-feature treatment cannot identify unusual combinations and primarily finds extreme values. COPOD extends empirical distributions into an empirical copula that estimates multivariate tail probabilities. section 6.3 section 6.4

Visualizing feature concentration

Load the breast-cancer dataset, select one feature, and plot histograms using 5, 20, and 200 bins alongside seaborn's ECDF. The ECDF remains free of a bin-count choice while revealing dense and sparse value ranges. section 6.3

How Outlier Detection in Python covers it

How deep it goes
Defines cumulative proportions, demonstrates plotting, and connects ECDFs to ECOD and COPOD without deriving their full internals.
Examples
Uses breast-cancer feature plots with seaborn and supplies ECOD and COPOD outlier explanations on speech and synthetic datasets.
What you can do after
Evaluate distributions with seaborn ECDF plots and understand how ECDF-based detectors score extreme observations.
How it connects
Connects empirical cumulative distributions with histograms, ECOD, COPOD, empirical copulas, outlier detection, and tail probabilities.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Outlier Detection in Python. The section links point to the parts of the book it draws on.