liveBook · Manning

Dimensionality reduction

As explained in Grokking Machine Learning, Second Edition

A data-transformation technique that reduces the number of features or dimensions while preserving as much relevant information as possible.

Dimensionality reduction simplifies a dataset by joining related features into fewer dimensions while preserving as much relevant information as possible. It is an unsupervised learning technique, so it can work without labels, and it is often used before another machine learning method. section 2.10 section 2.12

A smaller housing dataset

Imagine five housing features: size, number of bedrooms, number of bathrooms, neighborhood crime rate, and distance to the closest school. The first three describe the home’s size, while the last two describe neighborhood quality. A reduction algorithm could combine them into a general “size” feature and a general “neighborhood quality” feature, making the data easier to process and store without losing too much information. section 2.12

A dataset’s number of columns is its dimensionality: one column places points on a line, two columns place them in a city-like plane, and additional columns add further coordinates. Dimensionality reduction decreases the number of columns, whereas clustering simplifies data by reducing the number of rows through grouping. section 2.12 section 2.13

Figure 2.7 Dimensionality reduction algorithms help us simplify our data. On the left, we have a housing dataset with many features. We can use dimensionality reduction to reduce the number of features in the dataset without losing much information and obtain the dataset on the right.

Compare a housing dataset with many features to its reduced version containing fewer features while retaining much of the information. section 2.12

Matrix factorization and singular value decomposition extend the same simplification idea by reducing rows and columns together. Matrix factorization supports recommendation systems such as movie ratings, while singular value decomposition simplifies image data for compression. section 2.13 section 2.13

How Grokking Machine Learning, Second Edition covers it

How deep it goes
Explains feature grouping, dimensions, and related algorithms, but stops short of implementation details or mathematical procedures.
Examples
Uses housing features, similarity grouping, movie recommendations, and black-and-white image compression as examples.
What it assumes
Assumes familiarity only with dataset rows and columns, while explaining dimensions and feature grouping from intuitive examples.
What you can do after
Apply it as preprocessing to simplify datasets, reduce storage or processing demands, and retain much of their information.
How it connects
Connects unsupervised learning, clustering, preprocessing, matrix factorization, singular value decomposition, and supervised learning.

Also explained in

Compare how 15 books explain dimensionality reduction →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Grokking Machine Learning, Second Edition. The section links point to the parts of the book it draws on.