liveBook · Manning

Bootstrap sampling

As explained in Timeless Algorithms, The Foundational Ideas

A resampling technique that draws observations with replacement from an original dataset.

Bootstrap sampling creates a training sample by drawing observations from an original dataset with replacement. For a dataset of N observations, each sample contains N draws; observations may appear more than once, while others may be omitted. section 9.2.1 section 9.3.3

In bagging, each sample trains a model, and the models’ predictions are aggregated. Random forests use a new bootstrap sample for each tree, alongside random feature selection. Resampling creates variation, but does not guarantee that trees will differ enough to prevent correlated errors. section 9.2.1 section 9.2.3 section 9.3.3

How Timeless Algorithms, The Foundational Ideas covers it

How deep it goes
Explains drawing N observations with replacement and how duplicates and omissions arise.
What it assumes
Assumes an original dataset of N observations and defines resampling from that dataset.
What you can do after
Apply it by drawing N observations with replacement to create a training sample for each tree.
How it connects
Links bagging and random forests to decision trees by supplying each tree’s training observations.

Also explained in

Compare how 7 books explain bootstrap sampling →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Timeless Algorithms, The Foundational Ideas. The section links point to the parts of the book it draws on.