Diagonal matrix

As explained in Math and Architectures of Deep Learning

A square matrix whose only possibly nonzero entries are on the main diagonal.

A diagonal matrix is a square matrix whose only potentially nonzero entries lie on the main diagonal; every off-diagonal entry is zero. In diagonalization, the diagonal entries are eigenvalues, so the matrix records how a transformation scales independent eigenvector directions. section 2.15

Diagonal matrices make geometric and computational structure easier to read:

• For a symmetric matrix, orthogonal eigenvectors form the change-of-coordinate matrix S, giving A = SΛSᵀ, where Λ is diagonal. section 2.15
• In a quadratic-form equation, a diagonal matrix describes an axis-aligned ellipse or hyperellipse; rotating coordinates can reveal this form. section 2.17
• The diagonal entries preserve the directional scaling while removing cross-terms such as xy from the transformed ellipse equation. section 2.17.1

A rotated ellipse

The ellipse 5x² + 6xy + 5y² = 20 has matrix A = [[5, 3], [3, 5]]. Computing its eigenvectors with torch.linalg.eig produces a rotation S; in the rotated coordinates, SᵀAS is diagonal, and the major axis forms a 45° angle with the x-axis. section 2.17.1

Figure 2.16 Note that the ellipse’s major axis forms an angle of 45 degrees with the X-axis. Rotating the coordinate system by this angle will align the ellipse axes with the coordinate axes. Subsequently, the first principal vector will also lie along this direction.

Use the figure to connect diagonalization with the geometric alignment of a hyperellipse's axes. section 2.17.1

How Math and Architectures of Deep Learning covers it

How deep it goes

Defines the structure and works through eigenvector-based diagonalization and geometric transformation, without developing numerical stability details.

Examples

Uses the ellipse 5x² + 6xy + 5y² = 20, its matrix [[5,3],[3,5]], and a 45° eigenvector rotation.

What it assumes

Assumes matrices, eigenvectors, eigenvalues, transposes, dot products, and rotations.

What you can do after

Apply PyTorch eigenvalue computation to rotate a symmetric quadratic form into diagonal, axis-aligned coordinates.

How it connects

Connects diagonalization with eigenvalues, eigenvectors, quadratic forms, hyperellipses, PCA, and singular value decomposition.

liveBookmanning.com

Read Math and Architectures of Deep Learning on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Quantum Programming in DepthA diagonal matrix is a square matrix whose entries outside the main diagonal are zero.

Compare how 2 books explain diagonal matrix →

Related concepts

• degree matrixincludes · A degree matrix is a diagonal graph matrix whose entries record the number or total weight of connections incident to each node.
• arraykind of · An array is an indexed, ordered collection of elements arranged in one or more dimensions, with elements accessed by position.
• Gaussian eliminationproduced by · Gaussian elimination is a procedure that applies elementary row operations to transform a linear system or matrix into an equivalent triangular or row-reduced form.
• covariance matrixconfigured by · A covariance matrix is a square matrix whose diagonal entries are variances and whose off-diagonal entries are pairwise covariances among variables.

This explainer was generated by Manning's knowledge graph from the text of Math and Architectures of Deep Learning. The section links point to the parts of the book it draws on.