liveBook · Manning

Empirical risk

As explained in Timeless Algorithms, The Foundational Ideas

Empirical risk is the average loss a model incurs on the observations in a given sample.

Empirical risk is the average loss a model incurs on the observations in a sample. For ordinary classification accuracy, empirical misclassification risk equals one minus training accuracy. section 8.7.1

Empirial risk=1-training accuracy

For ordinary classification accuracy, this expresses empirical misclassification risk as one minus training accuracy; other loss functions are averaged over the sample. section 8.7.1

Empirical risk minimization (ERM) favors the permitted model with the lowest sample loss. That fit alone does not distinguish equally accurate models with different capacities; generalization depends on fit in relation to capacity and sample size. section 8.7.1 section 8.7.2

How Timeless Algorithms, The Foundational Ideas covers it

How deep it goes
Defines average sample loss and relates it to classification error, without developing a taxonomy of loss functions.
Examples
Uses training classification error and zero empirical misclassification risk for correctly classified hard-margin SVM data.
What you can do after
Evaluate training fit in relation to model capacity rather than treating low empirical risk as sufficient evidence of generalization.
How it connects
Connects to empirical risk minimization, structural risk minimization, VC dimension, generalization bounds, and support vector machines.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Timeless Algorithms, The Foundational Ideas. The section links point to the parts of the book it draws on.