Feature bagging

As explained in Outlier Detection in Python

Feature bagging is an ensemble technique that trains multiple detectors on randomly selected feature subsets and combines their outputs.

Feature bagging detects anomalies in small feature subspaces instead of evaluating every feature together. Full-dimensional distances can hide points unusual in only a few dimensions, while different anomalies may require different subspaces. section 6.10

A FeatureBagging ensemble proceeds by:

• Selecting a random number of features for each base detector, up to a configured maximum. section 6.11
• Drawing a bootstrap sample of rows for that detector. section 6.11
• Running a detector on the resulting subspace, using LOF by default but permitting other base detectors. section 6.11
• Combining each row’s detector scores with either their maximum or their mean; the maximum usually preserves subspace-specific outlier strength better. section 6.11

The method is useful when an observation is ordinary across most dimensions but anomalous in a small combination of features. However, selecting too many features can revive the curse of dimensionality and make base detectors correlated. section 6.10 section 6.11

Figure 6.14 The left pane shows point A in a 2D dataspace. The point is unusual considering feature A but not if using Euclidean distances in the full 2D dataspace. Using additional features can be counterproductive. In the middle and right panes, we see another point, point B, which is an outlier in the C–D subspace but not in the E–F subspace. We need only features C and D to identify this outlier.

Compare how the same points appear in different feature subspaces, including cases where additional dimensions obscure an anomaly. section 6.10

How Outlier Detection in Python covers it

How deep it goes

Explains subspace construction, base-detector scoring, score aggregation, and feature-count limitations without deriving detector internals.

Examples

Uses points anomalous in feature A or the C–D subspace to show why irrelevant dimensions can conceal outliers.

What you can do after

Apply it via PyOD by configuring randomized subspaces, base detectors, contamination, and score aggregation.

How it connects

Connects subspace outlier detection with LOF, bootstrap sampling, and ensemble methods.

liveBookmanning.com

Read Outlier Detection in Python on liveBook Get the book at manning.com

Related concepts

• local outlier factoruses · Local Outlier Factor is a density-based anomaly-detection algorithm that scores an observation by comparing its local density with those of its neighbors.
• subspaceuses · A subspace is a subset of a vector space that contains the zero vector and is closed under vector addition and scalar multiplication.
• random subspacesuses · Random subspaces is an ensemble technique that trains each model on a randomly selected subset of input features.
• subspace outlier detectionalternative to · Subspace outlier detection is an outlier detection method that identifies anomalies within feature subspaces selected from local reference sets.
• ensembleimplements · An ensemble is a technique that combines or considers outputs from multiple models or component methods collectively to produce or assess a joint result.

This explainer was generated by Manning's knowledge graph from the text of Outlier Detection in Python. The section links point to the parts of the book it draws on.