liveBook · Manning

L1 regularization

Also called Lasso

L1 regularization is a technique that adds a penalty proportional to the sum of absolute model-parameter values, encouraging sparse parameter estimates.

How 8 Manning books explain it

Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI

L1 regularization adds a penalty proportional to the sum of absolute model coefficients, encouraging sparse coefficient estimates.

How it is used
Selects useful predictors while reducing regression-model complexity and overfitting.
How deep it goes
Defines the absolute-coefficient penalty, explains sparsity and multicollinearity, and applies Lasso in a cross-validated pipeline.
Examples
Uses a polynomial tabular-regression pipeline with Lasso and cross-validation, plus bootstrapped stability selection with L1 logistic regression.
What you can do after
Apply it via scikit-learn Lasso or L1-penalized logistic regression, then evaluate sparse models with cross-validation.

Read how Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI explains L1 regularization →

Grokking Machine Learning

L1 regularization adds the L1 norm of a model's non-bias coefficients as a complexity penalty to its regression error.

How it is used
Reduces overfitting by penalizing model complexity and can zero out features when a dataset contains many.
How deep it goes
Defines the L1 penalty, explains coefficient shrinkage, and compares polynomial-model results without deriving the optimization procedure.
Examples
Compares regularized polynomial models using a 200-column dataset and contrasts their training and testing performance.
What you can do after
Apply L1 regularization through Turi Create’s linear_regression.create by setting l1_penalty.

Read how Grokking Machine Learning explains L1 regularization →

Grokking Machine Learning, Second Edition

L1 regularization penalizes the sum of the absolute values of model coefficients, encouraging some coefficients to become zero.

How it is used
Reduces overfitting while selecting among numerous regression features by driving many coefficients to zero.
How deep it goes
Explains the modified error, coefficient-shrinking intuition, and scikit-learn fitting, while deferring the mathematical justification for zero coefficients.
Examples
Uses movie-watch prediction, repeated shrinking of a coefficient from 2, and a degree-20 parabolic polynomial fitted with Lasso.
What you can do after
Apply it via scikit-learn Lasso by setting alpha, fitting a model, and comparing testing error with unregularized and Ridge models.

Read how Grokking Machine Learning, Second Edition explains L1 regularization →

Ensemble Methods for Machine Learning

L1 regularization penalizes the sum of absolute parameter values to constrain a model and encourage sparse parameters.

How it is used
Controls linear-regression complexity while encouraging sparse weights and implicit feature selection.
How deep it goes
Defines the absolute-value penalty and demonstrates its coefficient effect, but does not derive optimization internals.
Examples
Uses synthetic noisy data with two highly correlated features and compares coefficients, MSE, and mean absolute deviation.
What you can do after
Apply it via scikit-learn's Lasso estimator and evaluate predictions with MSE and mean absolute error.

Read how Ensemble Methods for Machine Learning explains L1 regularization →

Deep Learning with PyTorch, Second Edition

A regularization method that adds the sum of absolute weight values to the objective, encouraging sparse parameters.

How it is used
Uses a weight penalty to stabilize generalization and reduce overfitting in neural-network training.
How deep it goes
Defines the penalty and its sparse-weight effect, but does not provide an L1 implementation or optimization derivation.
What you can do after
Evaluate L1 regularization as a sparse-weight alternative to L2 regularization for controlling overfitting.

Read how Deep Learning with PyTorch, Second Edition explains L1 regularization →

Build AI Drug Discovery Pipelines

L1 regularization penalizes the sum of absolute model weights, encouraging some weights to become exactly zero.

How it is used
Uses coefficient sparsity to select molecular features and reduce overfitting in ligand-based and QSAR modeling.
How deep it goes
Defines the penalty and λ, then demonstrates scaling, grid search, cross-validation, coefficient filtering, and downstream regression.
Examples
Demonstrates descriptor selection for a roughly 48-ligand RNA-target QSAR dataset and compares selected regressions using R² and Q².
What you can do after
Apply it via scikit-learn to select descriptors, tune penalty strength, and evaluate predictions on held-out data.

Read how Build AI Drug Discovery Pipelines explains L1 regularization →

Machine Learning System Design

A regularization penalty based on the sum of absolute parameter values that promotes sparse model coefficients.

How it is used
Uses L1 regularization within Lasso regression for embedded feature selection.
How deep it goes
Explains coefficient elimination and method tradeoffs, but does not derive the penalty mathematically.
Examples
Illustrates selection with Lasso regression, where increasing regularization reduces coefficients to zero one by one.
What you can do after
Apply it via Lasso regression to prune features by identifying coefficients reduced to zero.

Read how Machine Learning System Design explains L1 regularization →

Deep Learning with Python, Third Edition

A regularization method that penalizes the absolute values of weights to encourage sparsity.

How it is used
Uses an added weight penalty to reduce overfitting during model development and validation-based tuning.
How deep it goes
Defines the absolute-value penalty and shows its Keras declaration, but does not work through L1 training behavior or results.
Examples
Uses Keras declarations for L1 and combined L1/L2 regularizers within a broader IMDb classification example.
What you can do after
Apply it via Keras by passing an L1 regularizer to a layer’s weight-regularizer argument.

Read how Deep Learning with Python, Third Edition explains L1 regularization →

Also covered in

What to know first

Related concepts

Questions about L1 regularization

What is L1 regularization?

L1 regularization is a technique that adds a penalty proportional to the sum of absolute model-parameter values, encouraging sparse parameter estimates.

Which Manning books explain L1 regularization?

Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI, Grokking Machine Learning, Grokking Machine Learning, Second Edition, Ensemble Methods for Machine Learning, Deep Learning with PyTorch, Second Edition, Build AI Drug Discovery Pipelines, Machine Learning System Design and Deep Learning with Python, Third Edition.

What should I know before learning L1 regularization?

Manning books that teach L1 regularization usually introduce overfitting, objective function, cross-validation and intercept first.

What is the difference between L1 regularization and ridge regression?

L1 regularization is a technique that adds a penalty proportional to the sum of absolute model-parameter values, encouraging sparse parameter estimates. Ridge regression is linear regression fitted with an L2 penalty on coefficient magnitudes, which shrinks estimates toward zero.

What is L1 regularization a kind of?

L1 regularization is a kind of regularization. Regularization is a modeling technique that constrains learning or penalizes model complexity to discourage overfitting and improve generalization.

What is L1 regularization also called?

L1 regularization is also called Lasso.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.