Structural risk minimization (SRM) selects a classifier by weighing empirical risk against model capacity. Greater flexibility is justified only when its improvement in training fit is sufficient to offset the increased risk of capturing peculiarities of the sample. section 8.7.1
Support vector machines put this balance into practice:
- Margin maximization controls boundary flexibility; soft-margin penalties balance violations against a wider margin. section 8.7.1
- Kernels allow more flexible boundary shapes when the data supports them. VC dimension provides a formal language for describing capacity. section 8.7.1
This relation expresses training error as the complement of training accuracy; SRM considers empirical risk together with capacity. section 8.7.1
