How deeply does Math and Architectures of Deep Learning cover singular value?

Defines singular values through SVD and AᵀA, proves nonnegativity, and applies them computationally without developing a full geometric proof.

Singular value

As explained in Math and Architectures of Deep Learning

A nonnegative quantity from singular value decomposition that measures how much a matrix stretches along an orthogonal direction.

A singular value measures how strongly a matrix stretches along an orthogonal direction. In the decomposition A = UΣVᵀ, the diagonal entries of Σ are the nonnegative square roots of the eigenvalues of AᵀA, ordered from largest to smallest. section 4.5 section 4.5.1

The associated right singular vectors are orthogonal directions in the input space. Because AᵀA is symmetric positive semidefinite, its eigenvalues—and therefore the singular values derived from them—are nonnegative. section 4.5.1

Reading singular values in latent semantic analysis

A dataset with correlated terms x₀ and x₁ produces singular values 4867.56982, 118.05858, and 19.68604. The much smaller third value indicates a weak variation that can be ignored, leaving two meaningful topic directions. section 4.6.4

Equation 4.11

A = UΣV^T

Use the decomposition equation to connect each singular value in Σ with the matrix transformation represented by UΣVᵀ. section 4.5

How Math and Architectures of Deep Learning covers it

Cover of Math and Architectures of Deep Learning

liveBook manning.com

Read Math and Architectures of Deep Learning on liveBook Get the book at manning.com

Also explained in

liveBook manning.com

Data Without LabelsA singular value is a nonnegative diagonal value in a singular value decomposition that quantifies the magnitude of a corresponding matrix direction.

Compare how 2 books explain singular value →

Related concepts

singular value decompositionproduced by · Singular value decomposition is a matrix factorization that expresses a matrix as the product of orthogonal or unitary matrices and a diagonal matrix of nonnegative singular values.
S / ATA / A^T A / A A^Tequivalent to · Matrices used in diagonalization and singular-value computations, including the matrix of eigenvectors S and the Gram matrices A^T A and A A^T.
matrix-matrix multiplicationequivalent to · Matrix multiplication combines compatible matrices to form a result whose entries are sums of products between rows of the first matrix and columns of the second.
eigenvalueequivalent to · An eigenvalue is a scalar by which a linear transformation scales a corresponding nonzero eigenvector without changing its direction.
standard deviationproduced by · Standard deviation is the nonnegative square root of variance and measures the typical dispersion of values around their mean in the original units.

This explainer was generated by Manning's knowledge graph from the text of Math and Architectures of Deep Learning. The section links point to the parts of the book it draws on.