liveBook · Manning

Regularization parameter

Also called penalty parameter

A regularization parameter is a model-fitting setting that controls the influence of a penalty or constraint relative to fit to the data.

How 4 Manning books explain it

Grokking Machine Learning

A training hyperparameter that controls the strength of the complexity penalty relative to predictive error.

How it is used
Balances predictive performance against model simplicity during training.
How deep it goes
Defines λ in the regularized error formula and explains zero and large values, but gives no detailed selection procedure.
Examples
Uses a λ = 0.1 polynomial-regression exercise to compare lasso and ridge errors under MAE.
What you can do after
Apply validation to help choose λ while balancing model simplicity and predictive performance.

Read how Grokking Machine Learning explains regularization parameter →

Timeless Algorithms, The Foundational Ideas

Regularization strength is a setting that controls how strongly a regularization penalty or constraint affects model fitting.

How it is used
Controls the cost of margin violations relative to margin width in soft-margin support vector machines.
How deep it goes
Defines C within the soft-margin objective and explains its effects on margin, violations, and underfitting.
Examples
Uses a noisy or mislabeled observation to contrast hard-margin fitting with tolerated soft-margin violations.
What you can do after
Apply k-fold cross-validation to compare candidate C values using held-out classification measures.

Read how Timeless Algorithms, The Foundational Ideas explains regularization parameter →

Grokking Machine Learning, Second Edition

A regularization parameter is a hyperparameter that controls the strength of the regularization penalty and the trade-off between model fit and complexity.

How it is used
Controls the balance between model performance and simplicity while training regression, classification, and neural-network models.
How deep it goes
Explains the penalty objective, coefficient-shrinkage procedures, L1/L2 differences, validation choices, and scikit-learn configuration.
Examples
Uses coefficient-shrinkage sequences, candidate powers of ten, a degree-20 polynomial dataset, and Lasso/Ridge testing errors.
What you can do after
Apply it via scikit-learn by configuring Lasso or Ridge alpha values and selecting among candidates with validation performance.

Read how Grokking Machine Learning, Second Edition explains regularization parameter →

Ensemble Methods for Machine Learning

A regularization parameter controls the strength of the complexity penalty relative to the data-fitting loss.

How it is used
Balances training fit against model simplicity during machine-learning optimization.
How deep it goes
Explains the fit-complexity tradeoff, parameter-direction effects, and validation-based tuning, but does not derive optimization procedures.
Examples
Uses SVR models, RBF kernels, Friedman-1 regression data, validation curves, and regularized logistic-loss formulas.
What you can do after
Apply it via scikit-learn validation curves to compare complexity, training fit, and generalization across parameter values.

Read how Ensemble Methods for Machine Learning explains regularization parameter →

What to know first

Related concepts

Questions about regularization parameter

What is regularization parameter?

A regularization parameter is a model-fitting setting that controls the influence of a penalty or constraint relative to fit to the data.

Which Manning books explain regularization parameter?

Grokking Machine Learning, Timeless Algorithms, The Foundational Ideas, Grokking Machine Learning, Second Edition and Ensemble Methods for Machine Learning.

What should I know before learning regularization parameter?

Manning books that teach regularization parameter usually introduce overfitting, objective function, support vector machine and underfitting first.

What is the difference between regularization parameter and margin?

A regularization parameter is a model-fitting setting that controls the influence of a penalty or constraint relative to fit to the data. In classification, a margin is a measure of separation between a decision boundary and data points, or of the correct class’s score advantage over competing classes.

What is regularization parameter a kind of?

Regularization parameter is a kind of hyperparameter. A hyperparameter is a setting chosen outside ordinary parameter learning that controls a model, algorithm, or learning procedure.

What is regularization parameter also called?

Regularization parameter is also called penalty parameter.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.