Regularization keeps a model from chasing noisy data. It balances prediction errors against model complexity, favoring simpler parameter settings that are less likely to overfit. section 3.16 section 7.2
In linear regression, regularization penalizes large weights. A model may use many features—such as a house’s size, location, and age—but discourages any one noisy feature from dominating the prediction. Minimizing prediction error and weight magnitude together produces a conservative fit. section 3.16
Bayesian models express the same preference through priors:
- A strong, zero-centered normal prior concentrates weight values near zero, reducing the influence of noisy features. section 3.16
- This produces the same effect as L₂ regularization in ordinary linear regression: both balance data fit against small weights. section 3.16
- With limited, noisy data, the prior prevents the data alone from pushing the model toward implausible conclusions. section 3.16
Sharing information across schools
Suppose separate school models estimate the effect of coaching on test scores. Pooling information across schools regularizes each school’s predictive distribution, reducing excessive variance and making extreme effects less likely. section 7.2
















