liveBook · Manning

Ensemble

As explained in Build AI Drug Discovery Pipelines

Ensemble learning combines multiple individual predictive models to form a composite model with improved robustness or accuracy.

Ensemble learning combines predictions from multiple individual models to improve accuracy and robustness. In drug discovery, it reduces the chance that one model's errors or biases determine a prediction. section 5.3.2

For decision trees, bagging trains trees on bootstrap samples and combines their outputs by voting for classification or averaging for regression. Random forests extend this strategy with diverse trees, reducing variance while limiting overfitting. section 5.3.2

A practical ensemble workflow includes:

  • Train multiple models on different sampled subsets of the training data. section 5.3.2
  • Combine classification predictions by voting or regression predictions by averaging. section 5.3.2
  • Use out-of-bag samples to estimate performance without a separate validation set when bootstrap training leaves observations unused for individual trees. section 5.3.2
  • Inspect feature importance, applicability-domain coverage, and performance inside versus outside the domain. section 5.3.2

CYP3A4 inhibition classifier

A RandomForestClassifier trained on molecular fingerprints achieved validation PR AUC 0.837, OOB accuracy 0.801, and OOB PR AUC 0.806 in a cytochrome P450 inhibition workflow. Fingerprint importance and similarity maps provide additional interpretation. section 5.3.2

Ensembling can also aggregate predictions across molecular conformers. Multiple-instance learning either combines conformer representations before prediction or combines predictions afterward; plurality voting suits classification, while averaging suits regression. section 5.3.3

Listing 5.5 Random Forest Training and Validation
from sklearn.ensemble import RandomForestClassifier
rf_clf = RandomForestClassifier(n_estimators=200, oob_score=True, n_jobs=-1)
rf_clf.fit(train_fingerprints, train_df.Inhibitor)
print(f"RF Validation PR AUC: "
      f"{pr_auc_score(rf_clf, val_fingerprints, val_df.Inhibitor):.3f}")
# Output -> RF Validation PR AUC: 0.837
 
print(f"RF OOB Accuracy: {rf_clf.oob_score_:.3f}")
# Output -> RF OOB Accuracy: 0.801
 
pred_train = np.argmax(rf_clf.oob_decision_function_, axis=1)
clf_precision, clf_recall, _ = metrics.precision_recall_curve(
  train_df.Inhibitor, pred_train
)
print(f"RF OOB PR AUC: {metrics.auc(clf_recall, clf_precision):.3f}")
# Output -> RF OOB PR AUC: 0.806
 
train_val_df = pd.concat([train_df, val_df])
rf_clf.fit(
  np.array(train_val_df["fingerprints"].values.tolist()),
  train_val_df.Inhibitor
)
print(f"Random Forest Test PR AUC: "
      f"{pr_auc_score(rf_clf, test_fingerprints, test_df.Inhibitor):.3f}")
# Output -> Random Forest Test PR AUC: 0.830

Runnable scikit-learn code trains a random forest, reports validation and out-of-bag metrics, and refits after combining training and validation data. section 5.3.2

How Build AI Drug Discovery Pipelines covers it

How deep it goes
Explains bootstrap sampling, voting, averaging, out-of-bag evaluation, interpretation, and conformer aggregation while deferring multiple-instance implementation.
Examples
Demonstrates CYP3A4 inhibition prediction with molecular fingerprints, out-of-bag metrics, feature importance, similarity maps, and conformer-level consensus.
What it assumes
Assumes basic molecular fingerprints and predictive modeling, while defining ensemble learning, decision trees, bootstrap sampling, and voting from fundamentals.
What you can do after
Build random-forest ensembles, evaluate them with out-of-bag estimates, and inspect their feature importance and applicability-domain behavior.
How it connects
Connects ensemble learning to decision trees, bagging, random forests, multiple-instance learning, applicability domains, and gradient boosting.

Also explained in

Compare how 16 books explain ensemble →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Build AI Drug Discovery Pipelines. The section links point to the parts of the book it draws on.