How deeply does Math and Architectures of Deep Learning cover unit vector?

Defines normalization and develops constrained quadratic-form optimization, then applies it to PCA and spectral norms.

Unit vector

As explained in Math and Architectures of Deep Learning

A vector whose length is exactly 1, used here to restrict the optimization to direction rather than magnitude.

A unit vector has length exactly 1. Dividing every element of a vector by its length produces the corresponding unit vector, which represents direction while removing magnitude. Unit vectors are conventionally written with a hat, such as x̂, and satisfy x̂ᵀx̂ = ||x̂||² = 1. section 2.5.5 section 4.2.1

This constraint makes direction the object of optimization. Without it, scaling a vector could make a quadratic form arbitrarily large or small; restricting x̂ to unit length instead asks which direction optimizes x̂ᵀAx̂. section 4.2.1

Unit vectors organize several geometric and machine-learning constructions:

Use a unit direction vector l̂ to obtain data components through dot products, l̂ᵀx. section 4.4.1
Find the maximum data spread by maximizing l̂ᵀCl̂; the maximizing unit vector is the eigenvector associated with the largest covariance-matrix eigenvalue. section 4.4.1
Measure a matrix’s greatest amplification by maximizing ||Ax̂|| over unit vectors; this leads to the spectral norm and singular values. section 4.3.1

Direction without magnitude

If a vector is normalized by dividing each element by its length, the result points in the same direction but has length 1. In PCA, the selected unit vector identifies a direction through the data; in matrix analysis, it identifies a direction that a matrix amplifies most or least. section 2.5.5 section 4.4.1 section 4.3.1

Equation 4.2

Q = x̂^TAx̂ = x̂^TSΛS^Tx̂ = ŷ^TΛŷ

Follow the quadratic-form transformation showing how a unit vector becomes coordinates in an eigenvector basis. section 4.2.1

How Math and Architectures of Deep Learning covers it

Cover of Math and Architectures of Deep Learning

liveBook manning.com

Read Math and Architectures of Deep Learning on liveBook Get the book at manning.com

Related concepts

vector normalizationproduced by · Vector normalization is the operation of scaling a nonzero vector to a specified norm, commonly unit length, while preserving its direction.
vectorkind of · A vector is an ordered, indexed sequence of elements, often numerical, represented as a mathematical quantity or a computational collection.
cosine similarityused by · Cosine similarity measures the directional alignment of two nonzero vectors as the cosine of the angle between them, equivalently their dot product divided by the product of their magnitudes.
l2 normuses · The L2 norm is the nonnegative Euclidean magnitude of a vector, equal to the square root of the sum of its components’ squares, and to absolute value for a scalar.
inner productused by · An inner product is a scalar-valued operation on two vectors that combines their corresponding components, summing their products and, for complex vectors, using conjugation.
directionuses · Direction is the orientation of a vector or parameter in space, considered independently of its magnitude.

This explainer was generated by Manning's knowledge graph from the text of Math and Architectures of Deep Learning. The section links point to the parts of the book it draws on.