liveBook · Manning

Noise

As explained in Outlier Detection in Python

Variation in data regarded as irrelevant or undesired within a particular context.

Noise is variation treated as irrelevant or undesired in a particular context. In outlier detection, it may mean borderline points between inliers and outliers, weak outliers worth flagging less strongly, or points to exclude without calling them full inliers. section 2.6 section 1.4

Noisy data often fades gradually from dense inliers into sparse regions, making the boundary between ordinary variation and strong outliers uncertain. A threshold therefore controls how much noise a detector flags, while different detectors may assign substantially different scores. section 2.6

Noise creates several practical decisions:

  • Separate strong outliers from borderline observations instead of treating every flagged point as equally meaningful. section 2.6
  • Interpret unusualness relative to context: variation regarded as noise in one setting may be signal in another. section 1.4
  • Treat irrelevant embedding dimensions as noise; standard tabular detectors such as Isolation Forest or KNN can perform poorly on high-dimensional embeddings. section 16.5.2

A gradual outlier boundary

Imagine a dense main cluster with points extending progressively farther to the right. The farthest points may safely be treated as outliers, but intermediate points have no obvious status; changing the threshold changes how much noise is flagged. section 2.6

Figure 1.2 Eight images of mugs. Each is different, and a reasonable argument could likely be made for any one of them to be considered anomalous relative to the other seven.

Compare the eight mug images and ask which one is anomalous; the example makes context-dependent judgment and uncertain boundaries concrete. section 1.4

How Outlier Detection in Python covers it

How deep it goes
Explains ambiguous categories, gradual density changes, threshold effects, score disagreement, and embedding-specific limitations.
Examples
Uses sparse points beyond a cluster, mug images, and irrelevant dimensions in embeddings as concrete cases.
What it assumes
Assumes comparison against a majority or generating process while defining contextual ambiguity from the outset.
What you can do after
Evaluate outlier flags by tuning thresholds and distinguishing strong outliers from contextual or weak noise.
How it connects
Connects noise with inliers, outliers, anomalies, outlier detection, thresholds, embeddings, Isolation Forest, and KNN.

Also explained in

Compare how 11 books explain noise →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Outlier Detection in Python. The section links point to the parts of the book it draws on.