The L2 norm measures a vector’s length or magnitude. For a vector x, ||x||₂ = √(x₁² + x₂² + ... + xₙ²): square each component, sum the squares, and take the square root. Its square, ||x||₂², equals the dot product x·x. section 2.5.4
Geometrically, the L2 norm of a two-dimensional vector is the hypotenuse of the right triangle formed by its components. The same length intuition extends to higher-dimensional vectors. section 2.5.5
The L2 norm supports several machine-learning interpretations:
- Treats the difference between a model’s output vector and target vector as an error vector whose length measures overall error. section 2.5.4
- Normalizes a nonzero vector by dividing every component by its length, producing a unit vector with length 1 that typically represents direction. section 2.5.5
- Provides a regularization penalty when added to a loss, discouraging neural networks from learning large weight values. section 9.3.6
Model error as vector length
Collect a model’s outputs and corresponding ground truths across a training set. Their differences form an error vector; minimizing its L2 norm minimizes the overall distance between predictions and targets. Using the squared L2 norm instead removes the final square root while preserving the dot-product interpretation. section 2.5.4 section 2.5.5
