L1 regularization adds the sum of the absolute model coefficients to regression error, penalizing complexity during training. This is also called lasso regression. The penalty can shrink coefficients to zero, making it useful when many features may be unnecessary. section 4.5 section 4.7
Lasso regression error = Regression error + L1 norm
The lasso objective combines regression error with the L1 norm; a regularization parameter can control the penalty’s weight. section 4.5
Unlike L2 regularization, which tends to make coefficients small without setting them to zero, L1 can eliminate coefficients. The choice depends on the goal: L1 is recommended for many features when turning some into zero is desirable. section 4.5 section 4.7
Polynomial regression with Turi Create
A polynomial model built from 200 columns is trained with l1_penalty=0.1 and l2_penalty=0. The L1 model performs reasonably on training and testing data, while the unregularized model fits training data well but generalizes poorly. section 4.6
