A regularization method that adds the sum of absolute weight values to the objective, encouraging sparse parameters.
L1 regularization combats overfitting by adding the sum of the absolute values of a model’s weights to the loss. A small hyperparameter controls the penalty, encouraging weights to become sparse. section 8.5.2
Use L1 regularization to:
Discourage individual weights from growing large while fitting training samples. section 8.5.2
Prefer models with many zero or near-zero parameters because the absolute-value penalty produces sparse trained weights. section 8.5.2
Compare it with L2 regularization, which penalizes the sum of squared weights instead. section 8.5.2
How Deep Learning with PyTorch, Second Edition covers it
How deep it goes
Defines the penalty and its sparse-weight effect, but does not provide an L1 implementation or optimization derivation.
What it assumes
Assumes familiarity with model weights, loss, training, and overfitting while introducing regularization penalties directly.
What you can do after
Evaluate L1 regularization as a sparse-weight alternative to L2 regularization for controlling overfitting.
How it connects
Connects regularization with overfitting, L2 regularization, weight penalties, and Lasso.
Ensemble Methods for Machine LearningL1 regularization penalizes the sum of absolute parameter values to constrain a model and encourage sparse parameters.
regularizationkind of · Regularization is a modeling technique that constrains or penalizes solutions during fitting or optimization to discourage undesirable complexity or behavior and improve generalization or estimation.
l2 normuses · The L2 norm is the nonnegative Euclidean magnitude of a vector, equal to the square root of the sum of its components’ squares, and to absolute value for a scalar.
feature selectionuses · Feature selection is the process of choosing a subset of input features for use in a modeling, analysis, or prediction task.
ridge regressioncontrasts with · Ridge regression is linear regression that applies an L2 penalty to coefficient magnitudes, shrinking estimates toward zero to reduce overfitting.
lassoused by · LASSO is a linear regression model that uses an L1 penalty to encourage some feature weights to become exactly zero.
XGBClassifiercontrasts with · XGBClassifier is XGBoost’s scikit-learn-compatible estimator for classification using gradient-boosted decision trees.
This explainer was generated by Manning's knowledge graph from the text of Deep Learning with PyTorch, Second Edition. The section links point to the parts of the book it draws on.
