liveBook · Manning

Random forest

As explained in Build AI Drug Discovery Pipelines

A random forest is an ensemble model that combines predictions from multiple decision trees trained with randomized data samples or feature selections.

A random forest combines predictions from multiple decision trees. It can classify molecules as inhibitors or non-inhibitors of a cytochrome P450 (CYP) enzyme. Such inhibition matters because it can change how other drugs are metabolized and contribute to drug–drug interactions. section 5.0 section 5.3.2

Each tree learns a series of feature-based decisions. The forest trains trees on bootstrap samples: random draws from the training data with replacement, so a sample can repeat some molecules and omit others. At each split, a tree also considers only a random subset of features. This extra randomness helps make the trees less alike. For classification, their predictions are combined by voting; for regression, they are averaged. Combining diverse trees can reduce the variance and overfitting associated with a single tree. section 5.3.2 section 5.5

Figure 5.9 How a Random Forest is constructed via a decision tree ensemble and applied at inference time.

Follow the construction of the tree ensemble and how its predictions are combined. section 5.3.2

Out-of-bag (OOB) evaluation uses training examples omitted from individual trees’ bootstrap samples. It provides an additional performance estimate without a separate validation set. Compare it with validation results using the same metric: accuracy and precision–recall area under the curve (PR AUC) measure different things. section 5.3.2

Predicting CYP inhibition

A scikit-learn RandomForestClassifier trains 200 trees on molecular fingerprints, numerical representations of molecules. The example reports validation PR AUC of 0.837 and OOB accuracy of 0.801. Its OOB PR AUC of 0.806 is calculated from predicted class labels, so it is not directly equivalent to a PR curve calculated from probability scores. section 5.3.2

Listing 5.5 Random Forest Training and Validation
from sklearn.ensemble import RandomForestClassifier
rf_clf = RandomForestClassifier(n_estimators=200, oob_score=True, n_jobs=-1)
rf_clf.fit(train_fingerprints, train_df.Inhibitor)
print(f"RF Validation PR AUC: "
      f"{pr_auc_score(rf_clf, val_fingerprints, val_df.Inhibitor):.3f}")
# Output -> RF Validation PR AUC: 0.837
 
print(f"RF OOB Accuracy: {rf_clf.oob_score_:.3f}")
# Output -> RF OOB Accuracy: 0.801
 
pred_train = np.argmax(rf_clf.oob_decision_function_, axis=1)
clf_precision, clf_recall, _ = metrics.precision_recall_curve(
  train_df.Inhibitor, pred_train
)
print(f"RF OOB PR AUC: {metrics.auc(clf_recall, clf_precision):.3f}")
# Output -> RF OOB PR AUC: 0.806
 
train_val_df = pd.concat([train_df, val_df])
rf_clf.fit(
  np.array(train_val_df["fingerprints"].values.tolist()),
  train_val_df.Inhibitor
)
print(f"Random Forest Test PR AUC: "
      f"{pr_auc_score(rf_clf, test_fingerprints, test_df.Inhibitor):.3f}")
# Output -> Random Forest Test PR AUC: 0.830

Inspect the classifier settings, training call, and validation and OOB calculations. section 5.3.2

A random forest can also predict multiple CYP inhibition labels for the same molecule. When combining datasets, missing inhibitor records should not automatically be treated as negative results: one workflow retains only molecules reported in all five datasets and uses stratified sampling to preserve label distributions. section 5.4.2

For interpretation, feature importance can rank fingerprint bits, while a molecular similarity map can show how parts of a molecule affect a predicted probability. A confidence threshold can also define an applicability domain, with performance checked separately inside and outside it. section 5.3.2

How Build AI Drug Discovery Pipelines covers it

How deep it goes
Explains bootstrap sampling, feature subsampling, aggregation, OOB evaluation, and classifier training, then demonstrates interpretation and applicability checks.
Examples
Trains a 200-tree CYP inhibitor classifier on molecular fingerprints and combines five CYP datasets for multilabel prediction.
What you can do after
Apply a random forest via scikit-learn to classify molecular fingerprints and evaluate validation and out-of-bag predictions.
How it connects
Connects decision trees and bagging with classification, multilabel learning, feature importance, and applicability domains.

Also explained in

Compare how 17 books explain random forest →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Build AI Drug Discovery Pipelines. The section links point to the parts of the book it draws on.