Imagine predicting how long someone will watch a movie from their viewing history. A model with many inputs can give every movie a coefficient, even when most are not useful. L1 regularization, also called lasso regression, adds a complexity penalty to regression error so training prefers a simpler model. The penalty is based on the absolute values of the coefficients. section 4.10 section 4.15
The key effect is feature selection. L1 regularization can make some coefficients exactly zero, removing those inputs from the final equation. For a recommendation model with 10 movie-history features, a long equation can become one using only x3, x7, and x9. This makes L1 a useful choice when there are many features and you want to discard most of them. section 4.12 section 4.15
L1 shrinks a positive coefficient by a fixed amount at each step. Starting at 2, repeated shrinking can reach zero; once zero, the coefficient stays there. By contrast, L2 repeatedly multiplies coefficients by a factor, making them smaller without reaching zero. section 4.13
Regularizing a high-degree curve
A degree-20 polynomial can fit training points extremely closely yet behave chaotically on unseen data. Split the data into training and testing sets, create Lasso with alpha=0.01, and fit it alongside an unregularized model. In the parabolic dataset, the L1 model fits the overall shape and has far lower testing error than the unregularized model. section 4.14
Use L1 regularization with a simple checklist:
- Start with prediction error, then add an L1 complexity term to the error being minimized. section 4.10
- Choose a regularization parameter: a penalty of zero means no regularization. section 4.14
- Compare training and testing performance; regularization aims to reduce overfitting rather than merely improve training fit. section 4.14 section 4.15
- Choose L2 instead when there are few features and all seem relevant, because L2 shrinks coefficients but rarely removes them. section 4.12
The same idea extends beyond linear regression. In a neural network, L1 adds the regularization parameter times the sum of absolute weight values to the error function, excluding biases. It is one way to punish high weights and reduce overfitting in regression, classification, and neural-network models. section 10.13







