Training fit alone cannot show whether a model will generalize. Cross-validation estimates performance on observations withheld from fitting, allowing candidate penalty values to be compared without evaluating them on their training data. section 8.3.4 section 8.3.5
In k-fold cross-validation, divide the data into k approximately equal folds and repeat these steps for each candidate value of C:
- Hold out one fold for validation and fit the model on the other k − 1 folds. section 8.3.5
- Evaluate the fitted model on the held-out fold using a suitable classification measure. section 8.3.5
- Rotate the held-out fold until every fold has served as validation data, then average the results. section 8.3.5
- Compare candidate values of C by their average validation performance. section 8.3.5
Five folds may suit larger data sets, while ten may be preferable with less data because each fit uses 90 rather than 80 percent of observations. Choose k with class balance, estimate variability, and computational cost in mind; each fold needs enough observations from both classes. Accuracy is one option, but balanced accuracy, precision, recall, or F1 may better suit imbalanced classes or unequal error consequences. section 8.3.5














