liveBook · Manning

L2 norm

As explained in Grokking Machine Learning, Second Edition

The L2 norm of a vector is its Euclidean length, computed as the square root of the sum of squared components.

The L2 norm is a way to put a number on a model’s complexity. Look at the model’s coefficients: models with more coefficients, or larger ones, tend to be more complex. Squaring each coefficient makes negative values positive, so large positive and negative coefficients cannot cancel each other out. In this treatment, the L2 norm is the sum of the squared coefficients, while the bias is left out because it is not treated as model complexity. section 4.9

Comparing two models

For coefficients 2, 1.4, and −0.5, the L2 norm is 2² + 1.4² + (−0.5)² = 6.21. A second model with ten much larger coefficients has an L2 norm of 227,131. The larger result reflects its greater complexity. section 4.9

L2 norm becomes useful when accuracy alone is not enough. Regularization adds a complexity measure to regression error, so training seeks a model that predicts well without becoming unnecessarily complicated. Using the L2 norm as that regularization term produces ridge regression. section 4.10 section 4.15

Ridge regression error = Regression error + L2 norm

This equation shows the ridge objective: regression error plus an L2-norm complexity term. section 4.10

During the intuitive coefficient-shrinking view of training, L2 regularization repeatedly multiplies a coefficient by a small factor. This makes coefficients smaller gradually; unlike L1 regularization, they do not reach zero through this process. L2 regularization is recommended when there are few features and the aim is to keep coefficients small rather than eliminate many of them. section 4.13 section 4.15

Try it with a regression model

Given a polynomial regression model, a regularization parameter λ = 0.1, and mean absolute error, calculate the ridge regression error by combining the regression error with its L2-norm term. Comparing that value with the lasso version makes the two complexity penalties concrete. section 4.16

How Grokking Machine Learning, Second Edition covers it

How deep it goes
Defines L2 norm through squared coefficients, excludes bias, and illustrates coefficient shrinkage without formal mathematics.
Examples
Works through two regression models’ squared-coefficient totals and assigns a ridge-error calculation exercise.
What it assumes
Introduces complexity from regression coefficients and relies on prior regression-error measures for the combined objective.
What you can do after
Apply it to calculate a ridge regression complexity term and favor small, nonzero coefficients with few features.
How it connects
Connects L2 norm to regularization, ridge regression, regression error, L1 norm, and model complexity.

Also explained in

Compare how 7 books explain l2 norm →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Grokking Machine Learning, Second Edition. The section links point to the parts of the book it draws on.