liveBook · Manning

Bagging

As explained in Build AI Drug Discovery Pipelines

Bagging is an ensemble technique that trains multiple instances of a learning algorithm on bootstrap-sampled subsets of data and aggregates their predictions.

Bagging, or bootstrap aggregating, combines predictions from multiple instances of one learning algorithm. Each instance trains on a different subset created by random sampling with replacement, then the ensemble averages regression outputs or votes on classification outputs. For decision trees, this reduces variance and makes predictions more stable and less prone to overfitting. section 5.3.2

A practical drug-discovery use case is classifying whether molecules inhibit Cytochrome P450 enzymes. CYP inhibition can alter the effects of other drugs and cause drug-drug interactions, making reliable classification useful during discovery. section 5.0

A bagging workflow follows these steps:

  • Create bootstrap samples from the training data, allowing observations to repeat while leaving others out. section 5.3.2
  • Train multiple instances of the same algorithm, such as decision trees, on the sampled subsets. section 5.3.2
  • Aggregate predictions by voting for classification or averaging for regression. section 5.3.2
  • Use out-of-bag samples—the observations omitted from each tree's bootstrap sample—to estimate performance without a separate validation set. section 5.3.2

CYP inhibition with fingerprint features

A RandomForestClassifier can train 200 trees with out-of-bag scoring enabled on molecular fingerprints. Validation PR AUC, OOB accuracy, and OOB PR AUC provide complementary checks before evaluating the final test set. section 5.3.2

Listing 5.5 Random Forest Training and Validation
from sklearn.ensemble import RandomForestClassifier
rf_clf = RandomForestClassifier(n_estimators=200, oob_score=True, n_jobs=-1)
rf_clf.fit(train_fingerprints, train_df.Inhibitor)
print(f"RF Validation PR AUC: "
      f"{pr_auc_score(rf_clf, val_fingerprints, val_df.Inhibitor):.3f}")
# Output -> RF Validation PR AUC: 0.837
 
print(f"RF OOB Accuracy: {rf_clf.oob_score_:.3f}")
# Output -> RF OOB Accuracy: 0.801
 
pred_train = np.argmax(rf_clf.oob_decision_function_, axis=1)
clf_precision, clf_recall, _ = metrics.precision_recall_curve(
  train_df.Inhibitor, pred_train
)
print(f"RF OOB PR AUC: {metrics.auc(clf_recall, clf_precision):.3f}")
# Output -> RF OOB PR AUC: 0.806
 
train_val_df = pd.concat([train_df, val_df])
rf_clf.fit(
  np.array(train_val_df["fingerprints"].values.tolist()),
  train_val_df.Inhibitor
)
print(f"Random Forest Test PR AUC: "
      f"{pr_auc_score(rf_clf, test_fingerprints, test_df.Inhibitor):.3f}")
# Output -> Random Forest Test PR AUC: 0.830

Inspect a runnable scikit-learn workflow that trains the ensemble, reports validation PR AUC and OOB metrics, and then refits using training and validation data. section 5.3.2

How Build AI Drug Discovery Pipelines covers it

How deep it goes
Defines bootstrap sampling, aggregation, variance reduction, and OOB evaluation, but stops short of a general bagging implementation.
Examples
Uses CYP inhibition fingerprints, RandomForestClassifier validation, OOB scores, and applicability-domain checks.
What it assumes
Builds from classification and decision trees before introducing ensemble learning and bootstrap sampling.
What you can do after
Apply it via scikit-learn's RandomForestClassifier and compare validation, OOB, and test-set performance.
How it connects
Links ensemble learning with decision trees, random forests, bootstrap sampling, and out-of-bag evaluation.

Also explained in

Compare how 6 books explain bagging →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Build AI Drug Discovery Pipelines. The section links point to the parts of the book it draws on.