Principal component analysis
Principal component analysis is a linear statistical technique that represents data along orthogonal directions, called principal components, ordered by the variance they capture.
Math and Architectures of Deep Learning
A multivariate statistical technique that finds orthogonal directions of maximal spread in data using covariance-matrix eigenvectors and eigenvalues.
- How it is used
- Reduces high-dimensional feature data to its dominant linear patterns for analysis, compression, and latent representations.
- How deep it goes
- Derives principal axes through covariance quadratic forms and eigenvectors, then implements direct and SVD-based PyTorch computations.
- Examples
- Uses correlated points near y = 2x, uncorrelated 2D data, nonlinear synthetic data, and 3D points near a plane.
- What you can do after
- Apply it via PyTorch eigendecomposition or SVD to project centered data, select components, and approximately reconstruct inputs.
Read how Math and Architectures of Deep Learning explains principal component analysis →
Data Without Labels
Principal component analysis is a linear dimensionality-reduction method that transforms correlated variables into ordered orthogonal components capturing decreasing amounts of variance.
- How it is used
- Reduces high-dimensional, correlated features before visualization or further machine learning analysis.
- How deep it goes
- Explains projection and orthogonality, calculates covariance eigenvalues and eigenvectors, and walks through a scikit-learn workflow.
- Examples
- Uses housing variables for intuition, Iris measurements for Python plots, and telecommunications subscribers for a business scenario.
- What you can do after
- Apply it via scikit-learn to standardized features and assess component counts with explained-variance plots.
Read how Data Without Labels explains principal component analysis →
Build AI Drug Discovery Pipelines
PCA is principal component analysis, a linear dimensionality-reduction method that projects data onto orthogonal directions of greatest variance.
- How it is used
- Reduces molecular features for chemical-space exploration and an RNA-binding QSAR workflow.
- How deep it goes
- Explains variance-ordered orthogonal components, projection, and component selection, including the subjectivity of scree plots.
- Examples
- Uses drug-molecule fingerprints and a two-dimensional projection example to illustrate PCA.
- What you can do after
- Apply it via scikit-learn to compress RDKit molecular fingerprints into four principal components.
Read how Build AI Drug Discovery Pipelines explains principal component analysis →
Natural Language Processing in Action, Second Edition
Principal component analysis is a statistical technique that transforms correlated variables into a smaller set of orthogonal components capturing as much variance as possible.
- How it is used
- Reduces dimensions for inspecting point clouds and visualizing word-vector relationships.
- How deep it goes
- Explains variance-maximizing components and SVD factorization, then compares centering and sparse-data costs with truncated SVD.
- Examples
- Uses a three-dimensional point cloud and 300-dimensional city word vectors projected into two dimensions.
- What you can do after
- Apply it via scikit-learn to fit and plot two-dimensional projections.
Outlier Detection in Python
Principal component analysis is a linear dimensionality-reduction technique that represents data using orthogonal directions of decreasing variance.
- How it is used
- Uses PCA for dimensionality reduction, visualization, and outlier detection in correlated feature data.
- How deep it goes
- Explains component interpretation and reconstruction-error scoring, then demonstrates Python workflows without a mathematical derivation.
- Examples
- Uses correlated two-feature outliers and synthetic large datasets to compare component choices, detector rankings, and runtimes.
- What you can do after
- Apply it via scikit-learn or PyOD to transform features, test components for outliers, or fit a reconstruction-based detector.
Read how Outlier Detection in Python explains principal component analysis →
Inside Deep Learning: Math, Algorithms, Models
Principal component analysis, a linear dimensionality-reduction method used here to project data into a lower-dimensional space.
- How it is used
- Projects feature vectors into lower-dimensional representations for dimensionality reduction and visualization, and connects this task to autoencoders.
- How deep it goes
- Recasts PCA as a linear encoder-decoder and implements shared transposed weights with reconstruction and orthogonality losses.
- Examples
- Uses MNIST images, a two-dimensional encoding, self-reconstruction targets, and a nonlinear autoencoder comparison.
- What you can do after
- Build it from scratch with PyTorch by sharing a linear layer’s weights with its transpose and training reconstruction plus orthogonality losses.
Read how Inside Deep Learning: Math, Algorithms, Models explains principal component analysis →
Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI
A linear dimensionality-reduction method that represents data using orthogonal components formed as weighted combinations of the original features.
- How it is used
- Uses dimensionality reduction to validate tabular data and visually inspect patterns or anomalies before modeling.
- How deep it goes
- Defines weighted linear components and their visualization role, but stops short of algorithmic or mathematical derivation.
- Examples
- Uses low-dimensional plots of tabular data to reveal patterns, isolated clusters, and potential anomalies.
- What you can do after
- Evaluate tabular data by creating compact visual summaries and recognizing when linear projections may be insufficient.
Machine Learning Algorithms in Depth
Principal component analysis is a dimensionality-reduction technique that projects data onto orthogonal directions of greatest variance.
- How it is used
- Reduces high-dimensional representations for inspection or compact image-search features.
- How deep it goes
- Derives covariance eigenvectors from a variance objective and implements PCA in NumPy; leaves runtime analysis as an exercise.
- Examples
- Uses a random-matrix projection and compresses ResNet50 encodings of Caltech 101 images before cosine-similarity retrieval.
- What you can do after
- Build it from scratch using covariance eigendecomposition to project data onto selected principal components.
Read how Machine Learning Algorithms in Depth explains principal component analysis →
Cyber Threat Hunting
Principal component analysis is a dimensionality-reduction method that projects data onto orthogonal components capturing decreasing amounts of variation.
- How it is used
- Uses dimensionality reduction to prepare correlated features for supervised machine-learning analysis.
- How deep it goes
- Explains the definition and workflow through scaling, variance plotting, transformation, and visualization without deriving PCA mathematically.
- Examples
- Demonstrates PCA with X_train_scaled, four components, cumulative-variance plots, and class-colored plots of PC pairs.
- What you can do after
- Apply it via scikit-learn to scale features, select component counts, transform training data, and visualize component relationships.
Read how Cyber Threat Hunting explains principal component analysis →
Evolutionary Deep Learning: Genetic algorithms and neural networks
Principal component analysis is a dimensionality-reduction technique that transforms correlated multidimensional data into a smaller set of components capturing variation in the data.
- How it is used
- Uses dimensionality reduction to visualize multidimensional hyperparameter-optimization results in two dimensions.
- How deep it goes
- Explains projection, variance-based components, and implementation, but defers detailed algorithmic treatment to documentation or other resources.
- Examples
- Uses four-dimensional contrived population data, scatter plots, and fitness-evaluation hexbin plots.
- What you can do after
- Apply it via scikit-learn to transform multidimensional optimization data and inspect search behavior through two-dimensional plots.
Graph Neural Networks in Action
Principal component analysis is a statistical dimensionality-reduction technique that transforms correlated variables into a smaller set of ordered orthogonal components.
- How it is used
- Reduces product-description features before using them as node inputs in graph-learning workflows.
- How deep it goes
- Explains PCA through its placement after bag-of-words and stops short of deriving the transformation mathematically.
- Examples
- Uses Amazon product descriptions, co-purchasing graphs, 100-feature vectors, product categorization, and link prediction.
- What you can do after
- Apply it via a preprocessing pipeline that converts bag-of-words product descriptions into 100-feature node representations.
Read how Graph Neural Networks in Action explains principal component analysis →
Human-in-the-Loop Machine Learning: Active learning and annotation for human-centered AI
Principal component analysis is a statistical technique that transforms correlated features into a smaller set of uncorrelated components.
- How it is used
- Reduces correlated feature dimensions for tasks such as diversity sampling and clustering.
- How deep it goes
- Defines PCA as combining highly correlated features, but defers implementation and mathematical derivation.
- Examples
- Uses a green car against a blue wall to illustrate how pixel-based image similarity can misrepresent image content.
- What you can do after
- Evaluate PCA alongside model embeddings when choosing a dimensionality-reduction approach for clustering.
Also covered in
- Privacy-Preserving Machine Learning
- AI Agents in Action, Second Edition
- Fast Python: High performance techniques for large datasets
- AI Agents in Action
- Ensemble Methods for Machine Learning
What to know first
- varianceVariance is the expected squared deviation of a random variable from its mean and measures the dispersion of its values.
- dimensionality reductionDimensionality reduction transforms data into a representation with fewer dimensions while attempting to retain relevant information or structure.
- anomalyAn anomaly is an observation, data state, or behavior that substantially deviates from an expected or prevailing pattern or condition.
- covariance matrixA covariance matrix is a square matrix whose diagonal entries are variances and whose off-diagonal entries are pairwise covariances among variables.
Related concepts
- dimensionality reductionkind of · Dimensionality reduction transforms data into a representation with fewer dimensions while attempting to retain relevant information or structure.
- covariance matrixuses · A covariance matrix is a square matrix whose diagonal entries are variances and whose off-diagonal entries are pairwise covariances among variables.
- varianceuses · Variance is the expected squared deviation of a random variable from its mean and measures the dispersion of its values.
- principal componentproduces · A principal component is an orthogonal direction in transformed data space that captures a ranked amount of variation in the data.
Questions about principal component analysis
What is principal component analysis?
Principal component analysis is a linear statistical technique that represents data along orthogonal directions, called principal components, ordered by the variance they capture.
Which Manning books explain principal component analysis?
Math and Architectures of Deep Learning, Data Without Labels, Build AI Drug Discovery Pipelines, Natural Language Processing in Action, Second Edition, Outlier Detection in Python, Inside Deep Learning: Math, Algorithms, Models, Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI, Machine Learning Algorithms in Depth and 4 more.
What should I know before learning principal component analysis?
Manning books that teach principal component analysis usually introduce variance, dimensionality reduction, anomaly and covariance matrix first.
What kinds of principal component analysis are there?
Kinds of principal component analysis include incremental principal component analysis.
What is principal component analysis a kind of?
Principal component analysis is a kind of dimensionality reduction. Dimensionality reduction transforms data into a representation with fewer dimensions while attempting to retain relevant information or structure.
What is principal component analysis also called?
Principal component analysis is also called PCA or principle component analysis.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.











