Bagging
Bagging, or bootstrap aggregation, is an ensemble technique that trains multiple models on bootstrap samples of data and combines their predictions.
Timeless Algorithms, The Foundational Ideas
Bagging, or bootstrap aggregation, is an ensemble technique that trains models on bootstrap samples and combines their predictions to reduce variability.
- How it is used
- Stabilizes predictions from unstable learners by combining models trained on varied samples.
- How deep it goes
- Explains bootstrap sampling and aggregation, their effect on variance, and the limit imposed by correlated errors.
- Examples
- Illustrates classification with 62 of 100 trees voting for one class and 38 for another.
- What you can do after
- Evaluate and understand how sampling and aggregation stabilize predictions and why shared errors limit the benefit.
Read how Timeless Algorithms, The Foundational Ideas explains bagging →
Ensemble Methods for Machine Learning
Bagging is an ensemble technique that trains multiple models independently and combines their predictions.
- How it is used
- Applies bootstrap-based parallel ensembling to stabilize predictions from homogeneous base learners.
- How deep it goes
- Works through sampling, fitting, voting, averaging, from-scratch trees, scikit-learn use, OOB evaluation, and parallelization.
- Examples
- Uses noisy two-dimensional classification, scikit-learn OOB scoring, parallel CPU training, and Wisconsin breast-cancer diagnosis evaluation.
- What you can do after
- Apply it via scikit-learn BaggingClassifier, including OOB scoring and parallel execution, or build a basic tree ensemble.
Read how Ensemble Methods for Machine Learning explains bagging →
Build AI Drug Discovery Pipelines
Bagging is an ensemble technique that trains multiple instances of a learning algorithm on bootstrap-sampled subsets of data and aggregates their predictions.
- How it is used
- Deploys ensemble learning for Cytochrome P450 inhibition classification in drug-discovery workflows.
- How deep it goes
- Defines bootstrap sampling, aggregation, variance reduction, and OOB evaluation, but stops short of a general bagging implementation.
- Examples
- Uses CYP inhibition fingerprints, RandomForestClassifier validation, OOB scores, and applicability-domain checks.
- What you can do after
- Apply it via scikit-learn's RandomForestClassifier and compare validation, OOB, and test-set performance.
Read how Build AI Drug Discovery Pipelines explains bagging →
Grokking Machine Learning
An ensemble technique that trains successive learners on random bootstrap subsets of data and aggregates their predictions.
- How it is used
- Combines randomly trained learners into a stronger model for classification or regression.
- How deep it goes
- Defines bootstrap sampling and prediction aggregation, then works through a random-forest example without deriving theoretical guarantees.
- Examples
- Uses a friends’ voting analogy and a spam-email dataset with lottery and sale word counts.
- What you can do after
- Apply bagging via Scikit-Learn’s RandomForestClassifier to fit a forest and score its predictions.
Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI
A collection of models trained on resampled data whose predictions are aggregated to reduce variance.
- How it is used
- Reduces prediction variance in tabular-data models through an averaging ensemble built from bootstrapped training samples.
- How deep it goes
- Explains sampling variants, aggregation rules, parameter settings, trade-offs, and a scikit-learn workflow without mathematical derivation.
- Examples
- Uses a bagged decision-tree classifier, preprocessing pipeline, sampling-strategy table, and reported accuracy and runtime.
- What you can do after
- Apply bagging via scikit-learn's BaggingClassifier or BaggingRegressor and tune sampling and estimator-count choices.
Read how Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI explains bagging →
Machine Learning Algorithms in Depth
Bagging is an ensemble technique that trains models on bootstrap-resampled data and aggregates their predictions to reduce variance.
- How it is used
- Reduces variance by averaging or voting across bootstrap-trained base learners.
- How deep it goes
- Derives the bootstrap-and-aggregate procedure and discusses ensemble size, learner stability, boundaries, and learning curves.
- Examples
- Uses decision-tree and KNN ensembles on iris data with subsampling, decision boundaries, cross-validation, and learning curves.
- What you can do after
- Apply it via scikit-learn’s BaggingClassifier and evaluate accuracy as the number of base estimators changes.
Read how Machine Learning Algorithms in Depth explains bagging →
Also covered in
What to know first
- decision treeA decision tree is a tree-structured model or representation in which successive tests or choices lead through branches to outcomes, predictions, or actions.
- ensembleAn ensemble is a technique or model that combines or considers outputs from multiple models, methods, or components to produce or assess a joint result.
- overfittingOverfitting is a modeling condition in which a model adapts too closely to particular data, including its noise or idiosyncrasies, and consequently generalizes poorly to new data.
- classifierA classifier is a predictive model that assigns an input to one or more discrete classes or categories.
Related concepts
- random forestused by · A random forest is an ensemble learning model that aggregates predictions from multiple decision trees trained with randomized data samples, feature subsets, or both.
- bootstrap samplinguses · Bootstrap sampling is a resampling technique that repeatedly draws observations with replacement from observed data to estimate a statistic’s sampling behavior or variability.
- modeuses · The statistical mode is the value or category occurring most frequently in a set of observations or outcomes.
- bootstrap sampleuses · A dataset formed by drawing observations with replacement from an original dataset.
- data aggregationuses · Data aggregation is the process of combining multiple data values, records, or observations into a consolidated or summarized result.
- decision treeuses · A decision tree is a tree-structured model or representation in which successive tests or choices lead through branches to outcomes, predictions, or actions.
Questions about bagging
What is bagging?
Bagging, or bootstrap aggregation, is an ensemble technique that trains multiple models on bootstrap samples of data and combines their predictions.
Which Manning books explain bagging?
Timeless Algorithms, The Foundational Ideas, Ensemble Methods for Machine Learning, Build AI Drug Discovery Pipelines, Grokking Machine Learning, Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI and Machine Learning Algorithms in Depth.
What should I know before learning bagging?
Manning books that teach bagging usually introduce decision tree, ensemble, overfitting and classifier first.
What is bagging also called?
Bagging is also called bootstrap aggregating.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.





