liveBook · Manning

Shared neighbors distance

As explained in Outlier Detection in Python

Shared nearest neighbors is a neighborhood-similarity method that measures similarity by the overlap between observations' nearest-neighbor sets.

Shared nearest neighbors (SNN) measures neighborhood similarity by counting how many nearest neighbors two observations have in common. Points generated by the same mechanism should be close and share many neighbors. section 6.10

SNN supports neighborhood construction when ordinary distances become unreliable:

  • Construct each point’s reference set from points with the most shared neighbors rather than the smallest Euclidean distances. section 6.10
  • Use the resulting neighborhood within SOD to identify a subspace whose features explain the greatest variance for that reference set. section 6.10
  • Rely on neighbor rank when high dimensionality or irrelevant features make specific distances less dependable. section 6.10

Correlated features in a 35-feature dataset

A synthetic dataset contains 100 records and 35 features; features 8 and 9 are correlated, while the others are irrelevant. One record is given an unusual combination of those correlated features. SOD uses a shared-neighbor reference set and identifies that record as the top outlier. section 6.10

How Outlier Detection in Python covers it

How deep it goes
Explains shared-neighbor intuition, reference-set construction, and high-dimensional robustness within SOD.
Examples
Uses a 35-feature synthetic dataset with correlated features 8 and 9 and one injected outlier.
What it assumes
Assumes nearest-neighbor and Euclidean-distance ideas, then relates neighbor rank to high-dimensional and irrelevant-feature problems.
What you can do after
Apply shared-neighbor neighborhoods through PyOD’s SOD detector to investigate subspace-specific outliers.
How it connects
Connects nearest-neighbor methods with KNN, LOF, SOD, subspace outlier detection, and Euclidean distance.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Outlier Detection in Python. The section links point to the parts of the book it draws on.