liveBook · Manning

Out-of-bag error

As explained in Build AI Drug Discovery Pipelines

Out-of-bag evaluation estimates an ensemble's performance using the bootstrap samples excluded from training each individual model.

Out-of-bag (OOB) evaluation estimates a Random Forest’s performance without a separate validation set. Bootstrap sampling leaves some training examples out of each tree’s sample; those excluded examples evaluate that tree, and the ensemble aggregates the resulting evaluations. section 5.3.2

This makes OOB estimates useful when data is limited and supports hyperparameter tuning or comparisons between ensemble configurations. In a CYP inhibition classifier, OOB accuracy is 0.801 and OOB PR AUC is 0.806. section 5.3.2

How Build AI Drug Discovery Pipelines covers it

How deep it goes
Explains bootstrap exclusion, per-tree evaluation, ensemble aggregation, and practical tuning use.
Examples
Uses a CYP inhibition classifier with molecular fingerprints, OOB accuracy, and OOB PR AUC.
What you can do after
Evaluate Random Forest performance and compare configurations using OOB metrics.
How it connects
Connects bootstrap sampling, bagging, decision trees, Random Forests, validation sets, and hyperparameter tuning.

Also explained in

Compare how 3 books explain out-of-bag error →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Build AI Drug Discovery Pipelines. The section links point to the parts of the book it draws on.