liveBook · Manning

Categorical outlier detector

As explained in Outlier Detection in Python

An outlier-detection model that analyzes features whose values belong to discrete categories.

A categorical outlier detector analyzes features whose values belong to discrete categories. It is useful when categorical data dominates and avoids the difficulties of creating numeric encodings, but relatively few such detectors exist. section 3.2.4

Frequent item set methods identify rows containing unusually few or infrequent combinations of values. Frequent pattern outlier factor (FPOF) scores a row from the frequent item sets it contains: inliers generally contain more and more frequent sets than outliers. Association rules provide another frequent-item-set approach. section 3.2.4

Apply categorical detection with these practical decisions:

  • Convert numeric and date columns into categories through binning when using a primarily categorical method; choose bins with interpretability and trade-offs in mind. section 3.2.4 section 3.3.2
  • For mixed tables, either numerically encode categorical columns for a numeric detector or bin numeric columns for a categorical detector; few detectors support both types directly. section 3.3.2
  • Interpret scores as evidence for inspection rather than automatic meaning: evaluation is difficult, often requiring manual review, representative data, and ongoing tuning. section 4.15
  • Entropy-based detectors score individual categorical cell values and combine them per row, without considering associations between features; they therefore emphasize univariate rarity rather than combinations. section 7.9

Expense and preference records

In staff expenses, repeated Staff ID–Account combinations form frequent patterns; a row containing substantially fewer or less frequent combinations can receive a higher outlier score. In a SpeedDating dataset, FPOF uses one-hot categorical values and a minimum-support threshold to find frequent item sets. Its most outlier row combines unusual preference ranges, while the most inlier row contains common combinations.

How Outlier Detection in Python covers it

How deep it goes
Explains FPOF, binning, mixed-data choices, and entropy, but stops short of a full association-rules treatment.
Examples
Uses staff expenses, binned expense values, SpeedDating preferences, frequent-item-set tables, and entropy-based test outputs.
What it assumes
Assumes tabular outlier-detection familiarity and teaches categorical representation through one-hot encoding, binning, and frequent patterns.
What you can do after
Apply FPOF after categorical preparation, choose binning or encoding for mixed data, and inspect flagged rows cautiously.
How it connects
Connects categorical detection with frequent item sets, FPOF, association rules, binning, one-hot encoding, entropy, and numeric detectors.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Outlier Detection in Python. The section links point to the parts of the book it draws on.