L1 regularization
L1 regularization is a technique that adds a penalty proportional to the sum of absolute model-parameter values, encouraging sparse parameter estimates.
Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI
L1 regularization adds a penalty proportional to the sum of absolute model coefficients, encouraging sparse coefficient estimates.
- How it is used
- Selects useful predictors while reducing regression-model complexity and overfitting.
- How deep it goes
- Defines the absolute-coefficient penalty, explains sparsity and multicollinearity, and applies Lasso in a cross-validated pipeline.
- Examples
- Uses a polynomial tabular-regression pipeline with Lasso and cross-validation, plus bootstrapped stability selection with L1 logistic regression.
- What you can do after
- Apply it via scikit-learn Lasso or L1-penalized logistic regression, then evaluate sparse models with cross-validation.
Grokking Machine Learning
L1 regularization adds the L1 norm of a model's non-bias coefficients as a complexity penalty to its regression error.
- How it is used
- Reduces overfitting by penalizing model complexity and can zero out features when a dataset contains many.
- How deep it goes
- Defines the L1 penalty, explains coefficient shrinkage, and compares polynomial-model results without deriving the optimization procedure.
- Examples
- Compares regularized polynomial models using a 200-column dataset and contrasts their training and testing performance.
- What you can do after
- Apply L1 regularization through Turi Create’s linear_regression.create by setting l1_penalty.
Read how Grokking Machine Learning explains L1 regularization →
Grokking Machine Learning, Second Edition
L1 regularization penalizes the sum of the absolute values of model coefficients, encouraging some coefficients to become zero.
- How it is used
- Reduces overfitting while selecting among numerous regression features by driving many coefficients to zero.
- How deep it goes
- Explains the modified error, coefficient-shrinking intuition, and scikit-learn fitting, while deferring the mathematical justification for zero coefficients.
- Examples
- Uses movie-watch prediction, repeated shrinking of a coefficient from 2, and a degree-20 parabolic polynomial fitted with Lasso.
- What you can do after
- Apply it via scikit-learn Lasso by setting alpha, fitting a model, and comparing testing error with unregularized and Ridge models.
Read how Grokking Machine Learning, Second Edition explains L1 regularization →
Ensemble Methods for Machine Learning
L1 regularization penalizes the sum of absolute parameter values to constrain a model and encourage sparse parameters.
- How it is used
- Controls linear-regression complexity while encouraging sparse weights and implicit feature selection.
- How deep it goes
- Defines the absolute-value penalty and demonstrates its coefficient effect, but does not derive optimization internals.
- Examples
- Uses synthetic noisy data with two highly correlated features and compares coefficients, MSE, and mean absolute deviation.
- What you can do after
- Apply it via scikit-learn's Lasso estimator and evaluate predictions with MSE and mean absolute error.
Read how Ensemble Methods for Machine Learning explains L1 regularization →
Deep Learning with PyTorch, Second Edition
A regularization method that adds the sum of absolute weight values to the objective, encouraging sparse parameters.
- How it is used
- Uses a weight penalty to stabilize generalization and reduce overfitting in neural-network training.
- How deep it goes
- Defines the penalty and its sparse-weight effect, but does not provide an L1 implementation or optimization derivation.
- What you can do after
- Evaluate L1 regularization as a sparse-weight alternative to L2 regularization for controlling overfitting.
Read how Deep Learning with PyTorch, Second Edition explains L1 regularization →
Build AI Drug Discovery Pipelines
L1 regularization penalizes the sum of absolute model weights, encouraging some weights to become exactly zero.
- How it is used
- Uses coefficient sparsity to select molecular features and reduce overfitting in ligand-based and QSAR modeling.
- How deep it goes
- Defines the penalty and λ, then demonstrates scaling, grid search, cross-validation, coefficient filtering, and downstream regression.
- Examples
- Demonstrates descriptor selection for a roughly 48-ligand RNA-target QSAR dataset and compares selected regressions using R² and Q².
- What you can do after
- Apply it via scikit-learn to select descriptors, tune penalty strength, and evaluate predictions on held-out data.
Read how Build AI Drug Discovery Pipelines explains L1 regularization →
Machine Learning System Design
A regularization penalty based on the sum of absolute parameter values that promotes sparse model coefficients.
- How it is used
- Uses L1 regularization within Lasso regression for embedded feature selection.
- How deep it goes
- Explains coefficient elimination and method tradeoffs, but does not derive the penalty mathematically.
- Examples
- Illustrates selection with Lasso regression, where increasing regularization reduces coefficients to zero one by one.
- What you can do after
- Apply it via Lasso regression to prune features by identifying coefficients reduced to zero.
Read how Machine Learning System Design explains L1 regularization →
Deep Learning with Python, Third Edition
A regularization method that penalizes the absolute values of weights to encourage sparsity.
- How it is used
- Uses an added weight penalty to reduce overfitting during model development and validation-based tuning.
- How deep it goes
- Defines the absolute-value penalty and shows its Keras declaration, but does not work through L1 training behavior or results.
- Examples
- Uses Keras declarations for L1 and combined L1/L2 regularizers within a broader IMDb classification example.
- What you can do after
- Apply it via Keras by passing an L1 regularizer to a layer’s weight-regularizer argument.
Read how Deep Learning with Python, Third Edition explains L1 regularization →
Also covered in
What to know first
- overfittingOverfitting is a modeling condition in which a model adapts too closely to particular data, including its noise or idiosyncrasies, and consequently generalizes poorly to new data.
- objective functionAn objective function is a mathematical function that assigns values to candidate solutions or parameter settings, which an optimization procedure seeks to minimize or maximize.
- cross-validationCross-validation is a model-evaluation technique that repeatedly partitions available data into training and held-out subsets to estimate performance on unseen data.
- interceptAn intercept is an additive constant parameter in a model that shifts its output independently of its input features.
Related concepts
- regularizationkind of · Regularization is a modeling technique that constrains learning or penalizes model complexity to discourage overfitting and improve generalization.
- feature selectionuses · Feature selection is the process of choosing a subset of input features for use in a modeling, analysis, or prediction task.
- l2 normuses · The L2 norm of a vector is its Euclidean length, calculated as the square root of the sum of the squares of its components.
- lassoused by · LASSO is a linear regression model that uses an L1 penalty to encourage some feature weights to become exactly zero.
- ridge regressioncontrasts with · Ridge regression is linear regression fitted with an L2 penalty on coefficient magnitudes, which shrinks estimates toward zero.
Questions about L1 regularization
What is L1 regularization?
L1 regularization is a technique that adds a penalty proportional to the sum of absolute model-parameter values, encouraging sparse parameter estimates.
Which Manning books explain L1 regularization?
Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI, Grokking Machine Learning, Grokking Machine Learning, Second Edition, Ensemble Methods for Machine Learning, Deep Learning with PyTorch, Second Edition, Build AI Drug Discovery Pipelines, Machine Learning System Design and Deep Learning with Python, Third Edition.
What should I know before learning L1 regularization?
Manning books that teach L1 regularization usually introduce overfitting, objective function, cross-validation and intercept first.
What is the difference between L1 regularization and ridge regression?
L1 regularization is a technique that adds a penalty proportional to the sum of absolute model-parameter values, encouraging sparse parameter estimates. Ridge regression is linear regression fitted with an L2 penalty on coefficient magnitudes, which shrinks estimates toward zero.
What is L1 regularization a kind of?
L1 regularization is a kind of regularization. Regularization is a modeling technique that constrains learning or penalizes model complexity to discourage overfitting and improve generalization.
What is L1 regularization also called?
L1 regularization is also called Lasso.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.







