Bagging, or bootstrap aggregating, combines predictions from multiple instances of one learning algorithm. Each instance trains on a different subset created by random sampling with replacement, then the ensemble averages regression outputs or votes on classification outputs. For decision trees, this reduces variance and makes predictions more stable and less prone to overfitting. section 5.3.2
A practical drug-discovery use case is classifying whether molecules inhibit Cytochrome P450 enzymes. CYP inhibition can alter the effects of other drugs and cause drug-drug interactions, making reliable classification useful during discovery. section 5.0
A bagging workflow follows these steps:
- Create bootstrap samples from the training data, allowing observations to repeat while leaving others out. section 5.3.2
- Train multiple instances of the same algorithm, such as decision trees, on the sampled subsets. section 5.3.2
- Aggregate predictions by voting for classification or averaging for regression. section 5.3.2
- Use out-of-bag samples—the observations omitted from each tree's bootstrap sample—to estimate performance without a separate validation set. section 5.3.2
CYP inhibition with fingerprint features
A RandomForestClassifier can train 200 trees with out-of-bag scoring enabled on molecular fingerprints. Validation PR AUC, OOB accuracy, and OOB PR AUC provide complementary checks before evaluating the final test set. section 5.3.2
Listing 5.5 Random Forest Training and Validation
from sklearn.ensemble import RandomForestClassifier
rf_clf = RandomForestClassifier(n_estimators=200, oob_score=True, n_jobs=-1)
rf_clf.fit(train_fingerprints, train_df.Inhibitor)
print(f"RF Validation PR AUC: "
f"{pr_auc_score(rf_clf, val_fingerprints, val_df.Inhibitor):.3f}")
# Output -> RF Validation PR AUC: 0.837
print(f"RF OOB Accuracy: {rf_clf.oob_score_:.3f}")
# Output -> RF OOB Accuracy: 0.801
pred_train = np.argmax(rf_clf.oob_decision_function_, axis=1)
clf_precision, clf_recall, _ = metrics.precision_recall_curve(
train_df.Inhibitor, pred_train
)
print(f"RF OOB PR AUC: {metrics.auc(clf_recall, clf_precision):.3f}")
# Output -> RF OOB PR AUC: 0.806
train_val_df = pd.concat([train_df, val_df])
rf_clf.fit(
np.array(train_val_df["fingerprints"].values.tolist()),
train_val_df.Inhibitor
)
print(f"Random Forest Test PR AUC: "
f"{pr_auc_score(rf_clf, test_fingerprints, test_df.Inhibitor):.3f}")
# Output -> Random Forest Test PR AUC: 0.830Inspect a runnable scikit-learn workflow that trains the ensemble, reports validation PR AUC and OOB metrics, and then refits using training and validation data. section 5.3.2





