Ridge regression
Ridge regression is linear regression that applies an L2 penalty to coefficient magnitudes, shrinking estimates toward zero to reduce overfitting.
Ensemble Methods for Machine Learning
Ridge regression is linear regression fitted with an L2 penalty on the coefficients to reduce model complexity.
- How it is used
- Stabilizes continuous-label linear regression when correlated features make unregularized coefficient estimates unreliable.
- How deep it goes
- Explains the L2 objective, coefficient shrinkage, α tradeoff, implementation, and evaluation, but does not develop optimization internals.
- Examples
- Uses a noisy synthetic dataset with two correlated features, scikit-learn model comparisons, and a bike-rental GLM baseline.
- What you can do after
- Apply it via scikit-learn, tune α, compare predictions with related regressors, and evaluate held-out performance using MSE, MAD, or R2.
Read how Ensemble Methods for Machine Learning explains ridge regression →
Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI
Ridge is a regularized linear regression estimator that penalizes the squared magnitude of coefficients.
- How it is used
- Uses coefficient shrinkage to stabilize linear regression when predictive features are multicollinear.
- How deep it goes
- Explains squared penalties, proportional shrinkage, coefficient behavior, and a preprocessing-to-evaluation pipeline.
- Examples
- Demonstrates a Scikit-learn degree-two polynomial pipeline and contrasts its coefficient behavior with Lasso.
- What you can do after
- Apply it via Scikit-learn Ridge inside a preprocessing pipeline and evaluate it with five-fold cross-validation.
Related concepts
- weight decayuses · Weight decay is a model-training regularization technique that discourages large parameter values by penalizing their magnitudes or shrinking them during optimization.
- regularizationkind of · Regularization is a modeling technique that constrains or penalizes solutions during fitting or optimization to discourage undesirable complexity or behavior and improve generalization or estimation.
- linear regressionequivalent to · Linear regression is a statistical modeling method that estimates a continuous response as a linear combination of predictor variables and fitted coefficients.
- L1 regularizationcontrasts with · L1 regularization is a technique that adds a penalty proportional to the sum of absolute model-parameter values, encouraging sparse parameter estimates.
- l2 normproduced by · The L2 norm is the nonnegative Euclidean magnitude of a vector, equal to the square root of the sum of its components’ squares, and to absolute value for a scalar.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.