Singular value decomposition
Singular value decomposition is a matrix factorization that expresses a matrix as the product of orthogonal or unitary matrices and a diagonal matrix of nonnegative singular values.
Data Without Labels
Singular value decomposition is a matrix factorization that expresses a matrix as the product of two orthogonal matrices and a diagonal matrix of singular values.
- How it is used
- Reduces high-dimensional data before visualization or further analysis.
- How deep it goes
- Defines the factorization, relates singular vectors to eigenvectors, and works through Python preparation and plots; defers mathematical proofs.
- Examples
- Uses encoded mushroom data, variance and class-colored plots, and a high-dimensional telecommunications subscriber scenario.
- What you can do after
- Apply it via NumPy to standardized numeric features and inspect singular-value variance shares and component plots.
Read how Data Without Labels explains singular value decomposition →
Math and Architectures of Deep Learning
The statement that any matrix can be decomposed as a product of orthogonal matrices and a diagonal matrix in singular value decomposition form.
- How it is used
- Supports PCA, LSA, and stable solution of square, overdetermined, and underdetermined linear systems.
- How deep it goes
- Derives the factorization through positive-semidefinite and orthogonality lemmas, then implements PCA and system solving in PyTorch.
- Examples
- Uses PyTorch inverse-versus-SVD solvers, pseudo-inverse comparison, direct-versus-SVD PCA, and a correlated document-term dataset for LSA.
- What you can do after
- Apply it via PyTorch to solve linear systems and compute PCA or LSA-related decompositions.
Read how Math and Architectures of Deep Learning explains singular value decomposition →
LLMs in Production
Singular value decomposition is a matrix factorization that expresses a matrix as a product of orthogonal and diagonal matrices.
- How it is used
- Explains matrix compression and supplies the low-rank mechanism behind LoRA adapters for parameter-efficient model adaptation.
- How deep it goes
- Defines U, Σ, and V, demonstrates reconstruction with NumPy and SciPy, and applies rank selection to LoRA without deriving SVD algorithms.
- Examples
- Uses a 4 × 4 NumPy matrix, SciPy svds reconstruction, and a 100 × 100 LoRA weight update with rank two.
- What you can do after
- Apply it via SciPy and NumPy to reconstruct a low-rank matrix and reason about LoRA rank, memory, and accuracy trade-offs.
Read how LLMs in Production explains singular value decomposition →
Natural Language Processing in Action, Second Edition
Singular value decomposition is a matrix factorization that expresses a matrix as the product of orthogonal singular-vector matrices and a diagonal singular-value matrix.
- How it is used
- Transforms sparse bag-of-words or TF–IDF matrices into topic vectors for dimensionality reduction, semantic analysis, and semantic search.
- How deep it goes
- Defines full and truncated factorization, explains sparsity and centering, and evaluates distance preservation without deriving the algorithm numerically.
- Examples
- Uses TF–IDF document matrices, a comment dataset, PCA comparisons, topic vectors, and pairwise distance measures.
- What you can do after
- Apply it via scikit-learn to reduce sparse text matrices and assess whether topic-vector distances retain document meaning.
Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI
A matrix factorization that expresses a matrix as the product of orthogonal matrices and a diagonal matrix of singular values.
- How it is used
- Uses numeric multivariate projections to validate tabular data and explore patterns or anomalies visually.
- How deep it goes
- Explains projection purpose and linear limitations but stops short of factorization mathematics or implementation.
- Examples
- Projects tabular observations into a few summary dimensions and plots them to inspect clusters and isolated points.
- What you can do after
- Evaluate numeric tabular data through lower-dimensional plots that support pattern, cluster, and anomaly inspection.
Grokking Machine Learning
A matrix decomposition that expresses a matrix as the product of orthogonal matrices and a diagonal matrix of singular values.
- How it is used
- Uses matrix decomposition to simplify data by reducing both its rows and columns, including for image compression.
- How deep it goes
- Explains the smaller-matrix idea and names an application, but gives no decomposition procedure.
- Examples
- Uses a black-and-white image represented by pixel intensities to illustrate image compression.
- What you can do after
- Understand how SVD can simplify a data matrix and support image compression.
Read how Grokking Machine Learning explains singular value decomposition →
Related concepts
- principal component analysisused by · Principal component analysis is a linear dimensionality-reduction technique that transforms data into orthogonal components ordered by the variance they capture.
- singular valueproduces · A singular value is a nonnegative scalar in a matrix’s singular value decomposition that measures stretching along a corresponding singular direction.
- co-occurrence matrixuses · A co-occurrence matrix is a table whose entries record the frequency or weight with which pairs of items occur together within specified contexts.
- dimensionality reductionuses · Dimensionality reduction is the transformation of data into a representation with fewer dimensions while retaining selected information, structure, or relationships.
- truncated singular value decompositionincludes · Truncated singular value decomposition is a matrix factorization technique that approximates a matrix using a selected number of singular components.
- compressionuses · Compression is the transformation of information into a more compact representation to reduce storage or transmission requirements while retaining information needed for recovery or use.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.