A regularization parameter controls how strongly a model trades prediction performance for simplicity. It is usually written as λ (lambda): multiply the regularization penalty by λ, then add it to the regression error. A value of 0 removes the penalty; a larger value favors a simpler model, which may fit the data less closely. section 4.11
The objective combines ordinary regression error with a penalty scaled by λ. section 4.11
Think of λ as a simplicity dial. Turn it down to let the model chase the training points; turn it up to shrink its coefficients and resist overfitting. L1 regularization subtracts a fixed amount from coefficients and can drive them to zero, while L2 repeatedly multiplies them by a factor slightly below one and usually leaves them nonzero. section 4.13 section 10.13
Choosing a value
Try candidate values such as 10, 1, 0.1, and 0.01, then choose the value producing the best validation performance. In a degree-20 polynomial example, an unregularized model fits the training points chaotically, while L1 and L2 models with parameter 0.01 generalize better; the reported testing RMSEs are 0.153 for L1 and 0.104 for L2. section 4.11 section 4.14
The parameter appears in several regularization workflows:
- Use λ = 0 to recover the model without regularization. section 4.11
- Use Lasso for L1 regularization or Ridge for L2 regularization. section 4.13
- In scikit-learn, provide the value as alpha when constructing Lasso or Ridge. section 4.14
- Tune regularization parameters alongside learning rate, epochs, architecture, and dropout probability in neural networks. section 10.16



