How deeply does Deep Learning with Python, Third Edition cover L1 regularization?

Defines the absolute-value penalty and shows its Keras declaration, but does not work through L1 training behavior or results.

L1 regularization

As explained in Deep Learning with Python, Third Edition

A regularization method that penalizes the absolute values of weights to encourage sparsity.

L1 regularization discourages overfitting by adding a cost proportional to each weight’s absolute value. It favors models whose weight distribution is more regular and can encourage sparse weights. section 5.4.4

Adding L1 regularization in Keras

Keras exposes L1 weight regularization through a layer’s regularizer argument, for example regularizers.l1(0.001). The same API also supports combining L1 and L2 penalties with l1_l2.

Use L1 regularization as a practical option when:

A small deep learning model is overfitting and needs an additional constraint on its weights. section 5.4.4
You are iterating on architectures and hyperparameters using validation performance, while avoiding repeated overfitting to the validation process. section 6.2.5

How Deep Learning with Python, Third Edition covers it

Cover of Deep Learning with Python, Third Edition

liveBook manning.com

Read Deep Learning with Python, Third Edition on liveBook Get the book at manning.com

Also explained in

liveBook manning.com

Machine Learning for Tabular Data: XGBoost, Deep Learning, and AIL1 regularization adds a penalty proportional to the sum of absolute model coefficients, encouraging sparse coefficient estimates.
liveBook manning.com

Grokking Machine LearningL1 regularization adds the L1 norm of a model's non-bias coefficients as a complexity penalty to its regression error.
liveBook manning.com

Ensemble Methods for Machine LearningL1 regularization penalizes the sum of absolute parameter values to constrain a model and encourage sparse parameters.
liveBook manning.com

Deep Learning with PyTorch, Second EditionA regularization method that adds the sum of absolute weight values to the objective, encouraging sparse parameters.
liveBook manning.com

Machine Learning System DesignA regularization penalty based on the sum of absolute parameter values that promotes sparse model coefficients.

Compare how 6 books explain L1 regularization →

Related concepts

regularizationkind of · Regularization is a model-fitting technique that uses constraints or penalties to discourage overfitting or undesirable complexity and improve generalization.
l2 normuses · The L2 norm is the nonnegative Euclidean magnitude of a vector, equal to the square root of the sum of its components’ squares, and to absolute value for a scalar.
feature selectionuses · Feature selection is the process of choosing a subset of input features for use in a modeling, analysis, or prediction task.
lassoused by · LASSO is a linear regression model that uses an L1 penalty to encourage some feature weights to become exactly zero.
ridge regressioncontrasts with · Ridge regression is linear regression that applies an L2 penalty to coefficient magnitudes, shrinking estimates toward zero to reduce overfitting.

This explainer was generated by Manning's knowledge graph from the text of Deep Learning with Python, Third Edition. The section links point to the parts of the book it draws on.