liveBook · Manning

Contamination parameter

As explained in Outlier Detection in Python

A contamination parameter specifies the expected proportion of outliers in a dataset, often to determine an outlier-classification threshold.

A contamination parameter specifies the fraction of records expected to be outliers. Detectors use it to convert numeric outlier scores into binary labels by setting a threshold for the required number of records. section 5.2.2

Interpret the parameter as a labeling control rather than as the detector’s scoring mechanism:

  • Set a fixed contamination fraction when the likely outlier proportion is known; for example, 0.01 flags one record among 100. section 6.10
  • Expect predict() to return binary results based on a threshold applied to numeric detector scores. section 5.2.2 section 6.1
  • Treat the default cautiously: it may derive a contamination rate by formula rather than assume a fixed fraction. section 5.2.2
  • Compare score rankings separately when ground truth exists, because changing contamination changes binary labels without necessarily changing the scores. section 5.2.2

Flagging one synthetic outlier

With 100 records, setting SOD contamination to 0.01 requests a single flagged outlier. The detector identifies an unusual combination in two correlated features, although results can change with the reference-set size and dimensionality. section 6.10

Listing 5.1 Example using Isolation Forest on the KDD Cup dataset
import numpy as np
import pandas as pd
from sklearn.datasets import fetch_kddcup99
from sklearn.ensemble import IsolationForest
from sklearn.preprocessing import OrdinalEncoder
from sklearn.metrics import confusion_matrix
from sklearn.metrics import roc_auc_score
import matplotlib.pyplot as plt
import seaborn as sns
 
np.random.seed(0)
 
X, y = fetch_kddcup99(subset="SA", percent10=True, random_state=42,
                      return_X_y=True, as_frame=True)
y = (y != b"normal.").astype(np.int32)
 
enc = OrdinalEncoder()                          #1
X = enc.fit_transform(X)
 
det = IsolationForest()                         #2
det.fit(X)
 
pred = pd.Series(det.predict(X))                #3
pred = pred.map({1: 0, -1: 1})
print(confusion_matrix(y, pred))                #4
 
pred = det.score_samples(X)                     #5
min_score = pred.min()
max_score = pred.max()
pred = [(x - min_score) / (max_score - min_score) for x in pred]
pred = [1.0 - x for x in pred]
roc_auc_score(y, pred)                          #6

Inspect the Isolation Forest workflow to see how detector scores, predictions, categorical encoding, and evaluation fit around contamination-based labeling. section 5.2.2

How Outlier Detection in Python covers it

How deep it goes
Explains fixed and default-derived rates, score thresholding, and evaluation, but does not derive the detectors’ scoring algorithms.
Examples
Uses the KDD Cup network dataset, synthetic clustered data, and a 35-feature SOD example with one inserted outlier.
What it assumes
Assumes a fitted outlier detector and introduces how its scores become labels when the likely outlier proportion is known or estimated.
What you can do after
Apply it via scikit-learn or PyOD to control binary outlier labels and compare those labels with numeric score rankings.
How it connects
Connects outlier detection with Isolation Forest, PyOD, anomaly scores, binary classification, and threshold selection.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Outlier Detection in Python. The section links point to the parts of the book it draws on.