How deeply does Outlier Detection in Python cover k-nearest neighbors?

Explains average-distance scoring, contrasts kth-neighbor scoring, and implements neighbor queries; leaves selection of a reliable k unresolved.

K-nearest neighbors

As explained in Outlier Detection in Python

A k-nearest-neighbors algorithm scores a record using its average distance to its k nearest neighboring records.

K-nearest neighbors (KNN) scores a record by its average distance to its k nearest neighboring records. A large score indicates that nearby records are relatively far away, making the record a candidate outlier—not necessarily a meaningful anomaly. Unlike a test built around one average and variance, neighbor distances can flag unusual values between clusters as well as values at the extremes. section 3.2.1 section 2.1.6

A related detector, kth-nearest neighbors, uses only the distance to the kth neighbor. Averaging distances across the k neighbors can be more robust. Both approaches extend from a single feature to multidimensional records; features must first be scaled so their distances are comparable. section 3.2.1

A practical KNN outlier check follows these steps:

Prepare and scale the features used to compare records. section 3.2.1 section 4.11
Use a neighbor-search structure such as scikit-learn’s BallTree to retrieve distances, then average each record’s returned distances to obtain a score. section 4.11 section 3.2.1
Inspect the score distribution, choose records to investigate, and examine their feature values and plots. Vary k rather than assuming one setting is reliable. section 4.11

What a high score may reveal

With two synthetic clusters, a KNN calculation using k = 3 assigns scores that can be displayed as point sizes, then marks the five or fifteen highest-scoring records. On prepared baseball data, a k = 4 detector chiefly flags extreme feature values; its score distribution alone does not establish that those are the most interesting rows. One- and two-feature plots help interpret the flags but may miss patterns in higher dimensions. section 3.2.1 section 4.11

Listing 4.6 Running KNN on the baseball data

from sklearn.neighbors import BallTree

tree = BallTree(df)
dist, ind = tree.query(df, k=4)
dist = [x.mean() for x in dist]
df['KNN Score'] = dist
df['KNN Outliers'] = df['KNN Score'] > 2.3

sns.histplot(df['KNN Score'])
plt.show()

The baseball implementation queries a BallTree with k = 4, averages the returned distances, and applies an example threshold to create labels for plotting. section 4.11

How Outlier Detection in Python covers it

liveBook manning.com

Read Outlier Detection in Python on liveBook Get the book at manning.com

Also explained in

liveBook manning.com

Causal Inference for Data ScienceK-nearest neighbors is a prediction method that uses the outcomes of the closest training observations to predict an input's outcome.
liveBook manning.com

Building LLM Applications with DSPyThe k examples most similar to a query according to a measure of similarity.

Compare how 3 books explain k-nearest neighbors →

Related concepts

KNN optimizerused by · A DSPy optimizer that selects demonstrations dynamically by retrieving the k training examples most similar to each language-model call's inputs.
radius detectoralternative to · A radius detector identifies or scores observations according to the number or arrangement of neighboring observations within a specified distance radius.
propensity score methodimplemented by · A propensity-score method is a causal-inference technique that estimates treatment effects by matching, weighting, stratifying, or otherwise adjusting units according to their observed treatment probabilities.
parameterconfigured by · A model parameter is a numerical value, such as a weight or bias, that is learned or set to determine how a model computes outputs.
lazy evaluationuses · Lazy evaluation is an evaluation strategy that defers computing an expression or value until its result is needed, potentially avoiding the computation entirely.

This explainer was generated by Manning's knowledge graph from the text of Outlier Detection in Python. The section links point to the parts of the book it draws on.