Ensemble learning combines predictions from multiple individual models to improve accuracy and robustness. In drug discovery, it reduces the chance that one model's errors or biases determine a prediction. section 5.3.2
For decision trees, bagging trains trees on bootstrap samples and combines their outputs by voting for classification or averaging for regression. Random forests extend this strategy with diverse trees, reducing variance while limiting overfitting. section 5.3.2
A practical ensemble workflow includes:
- Train multiple models on different sampled subsets of the training data. section 5.3.2
- Combine classification predictions by voting or regression predictions by averaging. section 5.3.2
- Use out-of-bag samples to estimate performance without a separate validation set when bootstrap training leaves observations unused for individual trees. section 5.3.2
- Inspect feature importance, applicability-domain coverage, and performance inside versus outside the domain. section 5.3.2
CYP3A4 inhibition classifier
A RandomForestClassifier trained on molecular fingerprints achieved validation PR AUC 0.837, OOB accuracy 0.801, and OOB PR AUC 0.806 in a cytochrome P450 inhibition workflow. Fingerprint importance and similarity maps provide additional interpretation. section 5.3.2
Ensembling can also aggregate predictions across molecular conformers. Multiple-instance learning either combines conformer representations before prediction or combines predictions afterward; plurality voting suits classification, while averaging suits regression. section 5.3.3
Listing 5.5 Random Forest Training and Validation
from sklearn.ensemble import RandomForestClassifier
rf_clf = RandomForestClassifier(n_estimators=200, oob_score=True, n_jobs=-1)
rf_clf.fit(train_fingerprints, train_df.Inhibitor)
print(f"RF Validation PR AUC: "
f"{pr_auc_score(rf_clf, val_fingerprints, val_df.Inhibitor):.3f}")
# Output -> RF Validation PR AUC: 0.837
print(f"RF OOB Accuracy: {rf_clf.oob_score_:.3f}")
# Output -> RF OOB Accuracy: 0.801
pred_train = np.argmax(rf_clf.oob_decision_function_, axis=1)
clf_precision, clf_recall, _ = metrics.precision_recall_curve(
train_df.Inhibitor, pred_train
)
print(f"RF OOB PR AUC: {metrics.auc(clf_recall, clf_precision):.3f}")
# Output -> RF OOB PR AUC: 0.806
train_val_df = pd.concat([train_df, val_df])
rf_clf.fit(
np.array(train_val_df["fingerprints"].values.tolist()),
train_val_df.Inhibitor
)
print(f"Random Forest Test PR AUC: "
f"{pr_auc_score(rf_clf, test_fingerprints, test_df.Inhibitor):.3f}")
# Output -> Random Forest Test PR AUC: 0.830Runnable scikit-learn code trains a random forest, reports validation and out-of-bag metrics, and refits after combining training and validation data. section 5.3.2















