liveBook · Manning

Model selection

As explained in Timeless Algorithms, The Foundational Ideas

Model selection is the process of choosing among candidate models based on their assumptions, representations, and expected behavior.

Parameter estimation fits a model; model selection chooses among plausible models to represent the underlying process. Since added parameters often improve fit to observed data, selection balances goodness of fit against complexity. section 3.7

A disciplined comparison keeps model choice separate from final evaluation:

  • Compare candidate models using criteria such as likelihood ratio tests, AIC, or BIC. section 3.7
  • For kernel configurations, fit each candidate on the same training folds and compare validation performance using a selected classification measure. section 8.6.3
  • Refit the selected configuration on all training data; keep the test set untouched until selection is complete. section 8.6.3

Choosing an SVM kernel configuration

For an RBF kernel, γ controls boundary responsiveness and interacts with the soft-margin penalty C. Large values of both can produce a highly responsive boundary; cross-validation compares candidate settings for likely generalization rather than choosing by training accuracy. section 8.6.3

How Timeless Algorithms, The Foundational Ideas covers it

How deep it goes
Balances fit against complexity and details cross-validation, while pointing to capacity and statistical learning theory for deeper explanation.
Examples
Compares RBF and polynomial kernel configurations, including interactions between γ, C, and polynomial degree.
What you can do after
Apply cross-validation to choose a kernel configuration, refit on training data, and reserve the test set for final evaluation.
How it connects
Links parameter estimation and maximum likelihood with likelihood ratio tests, AIC, BIC, cross-validation, and support vector machines.

Also explained in

Compare how 12 books explain model selection →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Timeless Algorithms, The Foundational Ideas. The section links point to the parts of the book it draws on.