liveBook · Manning

Empirical risk minimization

As explained in Timeless Algorithms, The Foundational Ideas

Empirical risk minimization selects a model from a permitted class by minimizing its average loss on observed data.

Empirical risk is the average loss on observed training data. ERM selects the permitted classifier with the lowest empirical risk; for ordinary classification accuracy, risk equals 1 − training accuracy. section 8.7.1

ERM does not distinguish classifiers with equal training loss, even if one boundary is more sensitive to individual observations. A hard-margin SVM addresses this by choosing the widest margin when several boundaries classify all training examples correctly. The soft-margin version allows violations, balancing them against a wider margin. section 8.7.1

Empirial risk=1-training accuracy

For ordinary classification accuracy, empirical risk equals one minus training accuracy. section 8.7.1

How Timeless Algorithms, The Foundational Ideas covers it

How deep it goes
Defines empirical risk and explains why ERM cannot distinguish equal-loss classifiers, then traces capacity-aware choices through hard- and soft-margin SVMs.
Examples
Uses classification accuracy, competing decision boundaries, and separable versus imperfectly separable training data.
What you can do after
Understand why training risk alone may not distinguish classifiers and how SVM margin selection addresses that limitation.
How it connects
Connects empirical risk to classification loss, training data, model capacity, and SVM margin selection.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Timeless Algorithms, The Foundational Ideas. The section links point to the parts of the book it draws on.