Dimensionality reduction

As explained in Outlier Detection in Python

Dimensionality reduction is the transformation of data into fewer dimensions while retaining relevant information.

Dimensionality reduction represents data using fewer dimensions while retaining information useful for the task. In outlier detection, it can make many-feature datasets faster to assess and sometimes improve detection. It can also make two- or three-dimensional visualization possible. The choice of what to retain matters: a compact representation is not automatically a good representation for finding outliers. section 10.1.1

Distance-based detectors can struggle as features accumulate. Differences that stand out in one feature may be swamped by distances across many others, leaving records roughly equidistant. Many features can also slow fitting or prediction and create apparently unusual values by chance. Reducing dimensions is one response; training separate detectors on feature subspaces is another. section 3.1.3 section 10.1

Principal component analysis (PCA) transforms correlated features into components. The first components capture more variance; later ones capture what remains. For points clustered along a diagonal in two-dimensional space, one component follows that pattern and the other runs across it. A point can therefore stand out either by being extreme along the pattern or by departing from it. Transforming data into the same number of components changes coordinates; selecting fewer components performs dimensionality reduction. section 6.9 section 10.1.1

A practical comparison should check detection as well as speed:

• Fit the same detectors on original features, all PCA components, and selected component subsets; compare fit and prediction times. section 10.1.1
• Compare outlier scores or ranks for cases of interest. Do not retain only the highest-variance components by habit: later components may expose departures from feature relationships. section 6.9 section 10.1.1
• Inspect whether the transformed view helps explain a flagged point. Separation in component space need not make the components themselves easy to interpret. section 10.1.1

Compare correlated and independent features

One test uses 100,000 rows and ten associated features, with a record that breaks their usual relationships. Another uses ten unassociated features, with a record made extreme in several. Six PyOD detectors are tested on the original data and PCA variations. In the correlated test, reducing to three components cuts LOF fit time from 19.4 to 1.4 and prediction time from 20.8 to 1.4 in the reported runs; not every detector gains equally. The known outlier ranks first for all six detectors when the last three components are used, illustrating why detection quality must be checked alongside runtime. section 10.1.1

Figure 10.1 Scatterplot of the last two components of the PCA transformation of the first dataset, which contained an unusual combination of values. Here we see a single point in the top-right of the space. It is clear that the point is a strong outlier, though it is not clear what components 8 and 9 represent.

The last two PCA components isolate a record with an unusual combination of values, while showing the difficulty of interpreting what those components represent. section 10.1.1

PCA is one option alongside t-SNE, UMAP, and auto-encoders. Different component selections can also supply varied views to an ensemble of outlier detectors. Choose the representation against the outliers sought, the detector’s runtime, and the need to explain results rather than treating one reduction as universally best. section 10.1.1 section 14.3.2

How Outlier Detection in Python covers it

How deep it goes

Works through PCA transformations and timed detector comparisons, but does not develop the internals of t-SNE, UMAP, or auto-encoders.

Examples

Tests correlated and independent 100,000-row datasets with planted outliers, PCA component subsets, and six PyOD detectors.

What it assumes

Assumes familiarity with tabular features and outlier detectors while illustrating PCA through a correlated-data example.

What you can do after

Apply it via scikit-learn PCA and PyOD detectors, comparing outlier ranks and runtime before choosing components.

How it connects

Connects the curse of dimensionality and PCA with distance-based outlier detection, feature subspaces, and ensembles.

liveBookmanning.com

Read Outlier Detection in Python on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Data Without LabelsDimensionality reduction is the transformation of data into fewer dimensions while retaining important structure or information.
• 

  liveBookmanning.com
  Math and Architectures of Deep LearningThe reduction of the effective number of dimensions in a representation by a transformation such as a singular weight matrix.
• 

  liveBookmanning.com
  Natural Language Processing in Action, Second EditionDimension reduction is the transformation of data into a representation with fewer features or dimensions while retaining relevant information.
• 

  liveBookmanning.com
  Grokking Machine LearningA technique that simplifies data by reducing or combining features while preserving as much information as possible.
• 

  liveBookmanning.com
  Graph Neural Networks in ActionDimensionality reduction is the transformation of data into a representation with fewer numerical dimensions while retaining selected information or structure.
• 

  liveBookmanning.com
  Machine Learning for Tabular Data: XGBoost, Deep Learning, and AIDimensionality reduction transforms data with many variables into a lower-dimensional representation while attempting to preserve important structure or information.
• 

  liveBookmanning.com
  Cyber Threat HuntingDimensionality reduction transforms data into a representation with fewer features while retaining important information.
• 

  liveBookmanning.com
  Graph Algorithms for Data ScienceDimensionality reduction is a technique that represents data with fewer numerical dimensions while preserving selected structure or information.
• 

  liveBookmanning.com
  Delivering Machine Learning Projects: From design to deploymentDimensionality reduction is the transformation of data into fewer dimensions while attempting to preserve informative structure.
• 

  liveBookmanning.com
  Inside Deep Learning: Math, Algorithms, ModelsThe process of mapping data from a higher-dimensional space into a lower-dimensional representation while retaining important structure.

Compare how 11 books explain dimensionality reduction →

Related concepts

• principal component analysisincludes · Principal component analysis is a linear dimensionality-reduction technique that transforms data into orthogonal components ordered by the variance they capture.
• UMAPincludes · UMAP, or Uniform Manifold Approximation and Projection, is a nonlinear manifold-learning technique that embeds high-dimensional data in a lower-dimensional space while preserving important neighborhood and structural relationships.
• t-SNEincludes · t-SNE is a nonlinear dimensionality-reduction technique that maps high-dimensional data to a lower-dimensional space while emphasizing local similarities or neighborhood relationships.
• curse of dimensionalityaddresses · The curse of dimensionality is the set of statistical, geometric, and computational difficulties that tend to increase as the number of dimensions grows.
• embeddinguses · A numerical vector representation of an item or input that encodes features or relationships useful for computational tasks.
• unsupervised learningkind of · Unsupervised learning is a machine-learning approach that discovers patterns, structure, or representations in data without labeled target outputs.

This explainer was generated by Manning's knowledge graph from the text of Outlier Detection in Python. The section links point to the parts of the book it draws on.