Cross-validation estimates how well a model generalizes without repeatedly reusing the held-out test set. In k-fold cross-validation, split the training data into k folds, train k models on k−1 folds, and validate each model on the remaining fold. Average the validation scores, then retrain the selected model on all training data before final test evaluation. section 3.3.3
Use cross-validation to compare models and tune configurations while preserving a clean final evaluation:
- Keep the test set sealed until the final model is chosen; repeated test-set use contaminates its estimate of unseen-data performance. section 3.3.3 section 3.4.2
- Choose folds carefully: random folds can place near-identical molecular analogs in both training and validation data, inflating apparent performance. section 3.3.3
- Report metrics suited to the task, such as accuracy, macro-precision, macro-recall, F1, or area under the precision–recall curve. section 3.3.3 section 6.4.1
- Use stratified folds when class representation matters, and average fold scores rather than trusting one partition. section 6.4.1 section 3.3.3
Five-fold molecular classification
With Scikit-Learn, call cross_validate(lin_cls, train_fingerprints, train_labels, cv=5) and average the returned validation accuracies. A five-fold run trains five models, each using four-fifths of the training fingerprints and validating on the remaining fifth; the reported mean accuracy is 0.7933463161311263. A dummy most-frequent classifier provides a baseline accuracy of 0.6938775510204082. section 3.3.3
Figure 3.13 k-fold cross validation to evaluate model candidates on training data split into train and validation folds. The best model is retrained on all training data and evaluated on the held-out test data. Not shown, but before deploying the best model to a production environment, we would also retrain it on all available data, including the test set.

The fold diagram shows model candidates evaluated on training folds, the selected model retrained on all training data, and the untouched test evaluation. section 3.3.3














