Empirical risk is the average loss on observed training data. ERM selects the permitted classifier with the lowest empirical risk; for ordinary classification accuracy, risk equals 1 − training accuracy. section 8.7.1
ERM does not distinguish classifiers with equal training loss, even if one boundary is more sensitive to individual observations. A hard-margin SVM addresses this by choosing the widest margin when several boundaries classify all training examples correctly. The soft-margin version allows violations, balancing them against a wider margin. section 8.7.1
For ordinary classification accuracy, empirical risk equals one minus training accuracy. section 8.7.1
