Dimensionality reduction

As explained in Inside Deep Learning: Math, Algorithms, Models

The process of mapping data from a higher-dimensional space into a lower-dimensional representation while retaining important structure.

Dimensionality reduction maps a feature vector from D dimensions into a smaller D′-dimensional representation while preserving important structure. PCA provides a classic approach; autoencoders extend the same encoder–decoder idea with larger, more expressive networks. section 7.1.1 section 7.2

PCA can be viewed as a bottleneck autoencoder: a weight matrix encodes, its transpose decodes, and a reconstruction objective uses squared error with a constraint-related regularization penalty. Autoencoders commonly taper toward a bottleneck and expand back toward the input size. section 7.1.1 section 7.2

Reconstruction-based outlier detection

Train an autoencoder, compute reconstruction errors, and flag the highest-error cases using a threshold matched to the expected abnormal fraction. Poor reconstruction suggests that an input is unusual and may need review rather than prediction. section 7.2

Figure 7.4 Example of a standard autoencoder design. The input comes in on the left. The encoder starts out large and tapers down the size of the hidden layers at each step. Because autoencoders are usually symmetric, our decoder will receive the small representation z and begin expanding it back to the original size.

Shows the standard bottleneck layout: the encoder contracts toward representation z, while the decoder expands back toward the original input size. section 7.2

How Inside Deep Learning: Math, Algorithms, Models covers it

How deep it goes

Defines PCA through encoder-decoder notation, weight sharing, reconstruction loss, and constraints, then extends the idea to multilayer bottleneck networks.

Examples

Uses D-dimensional feature vectors, bottleneck representations z, symmetric networks, and reconstruction-error thresholds.

What you can do after

Apply via PCA or an autoencoder, and diagnose unusual inputs through reconstruction error.

How it connects

Connects PCA with autoencoders, encoder-decoder architectures, weight sharing, reconstruction error, and outlier detection.

liveBookmanning.com

Read Inside Deep Learning: Math, Algorithms, Models on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Data Without LabelsDimensionality reduction is the transformation of data into fewer dimensions while retaining important structure or information.
• 

  liveBookmanning.com
  Math and Architectures of Deep LearningThe reduction of the effective number of dimensions in a representation by a transformation such as a singular weight matrix.
• 

  liveBookmanning.com
  Natural Language Processing in Action, Second EditionDimension reduction is the transformation of data into a representation with fewer features or dimensions while retaining relevant information.
• 

  liveBookmanning.com
  Outlier Detection in PythonDimensionality reduction is the transformation of data into fewer dimensions while retaining relevant information.
• 

  liveBookmanning.com
  Grokking Machine LearningA technique that simplifies data by reducing or combining features while preserving as much information as possible.
• 

  liveBookmanning.com
  Graph Neural Networks in ActionDimensionality reduction is the transformation of data into a representation with fewer numerical dimensions while retaining selected information or structure.
• 

  liveBookmanning.com
  Machine Learning for Tabular Data: XGBoost, Deep Learning, and AIDimensionality reduction transforms data with many variables into a lower-dimensional representation while attempting to preserve important structure or information.
• 

  liveBookmanning.com
  Cyber Threat HuntingDimensionality reduction transforms data into a representation with fewer features while retaining important information.
• 

  liveBookmanning.com
  Graph Algorithms for Data ScienceDimensionality reduction is a technique that represents data with fewer numerical dimensions while preserving selected structure or information.
• 

  liveBookmanning.com
  Delivering Machine Learning Projects: From design to deploymentDimensionality reduction is the transformation of data into fewer dimensions while attempting to preserve informative structure.

Compare how 11 books explain dimensionality reduction →

Related concepts

• principal component analysisincludes · Principal component analysis is a linear dimensionality-reduction technique that transforms data into orthogonal components ordered by the variance they capture.
• UMAPincludes · UMAP, or Uniform Manifold Approximation and Projection, is a nonlinear manifold-learning technique that embeds high-dimensional data in a lower-dimensional space while preserving important neighborhood and structural relationships.
• t-SNEincludes · t-SNE is a nonlinear dimensionality-reduction technique that maps high-dimensional data to a lower-dimensional space while emphasizing local similarities or neighborhood relationships.
• curse of dimensionalityaddresses · The curse of dimensionality is the set of statistical, geometric, and computational difficulties that tend to increase as the number of dimensions grows.
• embeddinguses · A numerical vector representation of an item or input that encodes features or relationships useful for computational tasks.
• unsupervised learningkind of · Unsupervised learning is a machine-learning approach that discovers patterns, structure, or representations in data without labeled target outputs.

This explainer was generated by Manning's knowledge graph from the text of Inside Deep Learning: Math, Algorithms, Models. The section links point to the parts of the book it draws on.