How deeply does Math and Architectures of Deep Learning cover quadratic form?

Defines the form, rewrites circles and ellipses, and works through eigenvalue-based minimization; it stops short of full derivations beyond the stated lemmas.

Quadratic form

As explained in Math and Architectures of Deep Learning

A homogeneous degree-two expression, here written with a symmetric matrix as x^T A x or equivalent forms describing geometric shapes and curvature.

A quadratic form is a scalar built from a vector and a symmetric matrix, written as x^T A x. It describes circles, ellipses, hyperellipsoids, and curvature in matrix form. section 4.2 section 4.0

With symmetric A, the same expression can represent a hypersphere when A is diagonal, or a rotated hyperellipsoid when off-diagonal terms appear. Rotating the coordinate system diagonalizes the shape. section 4.2 section 2.17.1

Key uses here include:

finding principal axes by diagonalizing A. section 4.2.1
measuring variance along a direction as l^T C l. section 4.7
expressing the Hessian term in a multidimensional Taylor expansion. section 4.2
computing the spectral norm through x̂^T A^T A x̂. section 4.3.1

Ellipse example

The equation 5x^2 + 6xy + 5y^2 = 20 is rewritten as a matrix quadratic form. Diagonalizing the symmetric matrix reveals eigenvectors at 45°, which align with the ellipse axes. section 2.17.1

Listing 2.21 Axes of a hyperellipse

ellipse_eq = sy.Eq(5*x**2 + 5*y**2 +
                      6*x*y, 20)                   ①

A = torch.tensor([[5, 3], [3, 5]]).float()

l, S = torch.linalg.eig(A)

x_axis_vec = torch.zeros((A.shape[0]))             ②

first_eigen_vec = S[:, 0]                          ③

dot_prod = torch.dot(x_axis_vec, first_eigen_vec)  ④

theta = math.acos(dot_prod)
theta = math.degrees(theta)                        ⑤

① Equation of the ellipse: 5x² + 6xy + 5y² = 20 or ^TA = 20, where

② X-axis vector

③ Major axis of the ellipse

④ The dot product between two vectors is the cosine of the angle between them.

⑤ The angle between the ellipse’s major axis and the X-axis: 45° (see figure 2.16)

Shows the ellipse equation, matrix form, eigenvector-based axis direction, and the 45° angle calculation. section 2.17.1

Equation 4.2

Q = x̂^TAx̂ = x̂^TSΛS^Tx̂ = ŷ^TΛŷ

Shows the unit-vector form after diagonalization, making the eigenvalue weighting explicit. section 4.2.1

How Math and Architectures of Deep Learning covers it

Cover of Math and Architectures of Deep Learning

liveBook manning.com

Read Math and Architectures of Deep Learning on liveBook Get the book at manning.com

Related concepts

covariance matrixused by · A covariance matrix is a square matrix whose diagonal entries are variances and whose off-diagonal entries are pairwise covariances among variables.
eigenvectoruses · An eigenvector is a nonzero vector that a linear transformation maps to a scalar multiple of itself.
principal component analysisused by · Principal component analysis is a linear dimensionality-reduction technique that transforms data into orthogonal components ordered by the variance they capture.
eigenvalueuses · An eigenvalue is a scalar by which a linear transformation scales a corresponding nonzero eigenvector without changing its direction.

This explainer was generated by Manning's knowledge graph from the text of Math and Architectures of Deep Learning. The section links point to the parts of the book it draws on.