Bagging
Bagging, or bootstrap aggregation, is an ensemble technique that trains multiple models on bootstrap samples of data and combines their predictions.
Ensemble Methods for Machine Learning
Bagging is an ensemble technique that trains multiple models independently and combines their predictions.
- How it is used
- Applies bootstrap-based parallel ensembling to stabilize predictions from homogeneous base learners.
- How deep it goes
- Works through sampling, fitting, voting, averaging, from-scratch trees, scikit-learn use, OOB evaluation, and parallelization.
- Examples
- Uses noisy two-dimensional classification, scikit-learn OOB scoring, parallel CPU training, and Wisconsin breast-cancer diagnosis evaluation.
- What you can do after
- Apply it via scikit-learn BaggingClassifier, including OOB scoring and parallel execution, or build a basic tree ensemble.
Read how Ensemble Methods for Machine Learning explains bagging →
Grokking Machine Learning
An ensemble technique that trains successive learners on random bootstrap subsets of data and aggregates their predictions.
- How it is used
- Combines randomly trained learners into a stronger model for classification or regression.
- How deep it goes
- Defines bootstrap sampling and prediction aggregation, then works through a random-forest example without deriving theoretical guarantees.
- Examples
- Uses a friends’ voting analogy and a spam-email dataset with lottery and sale word counts.
- What you can do after
- Apply bagging via Scikit-Learn’s RandomForestClassifier to fit a forest and score its predictions.
Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI
A collection of models trained on resampled data whose predictions are aggregated to reduce variance.
- How it is used
- Reduces prediction variance in tabular-data models through an averaging ensemble built from bootstrapped training samples.
- How deep it goes
- Explains sampling variants, aggregation rules, parameter settings, trade-offs, and a scikit-learn workflow without mathematical derivation.
- Examples
- Uses a bagged decision-tree classifier, preprocessing pipeline, sampling-strategy table, and reported accuracy and runtime.
- What you can do after
- Apply bagging via scikit-learn's BaggingClassifier or BaggingRegressor and tune sampling and estimator-count choices.
Read how Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI explains bagging →
Machine Learning Algorithms in Depth
Bagging is an ensemble technique that trains models on bootstrap-resampled data and aggregates their predictions to reduce variance.
- How it is used
- Reduces variance by averaging or voting across bootstrap-trained base learners.
- How deep it goes
- Derives the bootstrap-and-aggregate procedure and discusses ensemble size, learner stability, boundaries, and learning curves.
- Examples
- Uses decision-tree and KNN ensembles on iris data with subsampling, decision boundaries, cross-validation, and learning curves.
- What you can do after
- Apply it via scikit-learn’s BaggingClassifier and evaluate accuracy as the number of base estimators changes.
Read how Machine Learning Algorithms in Depth explains bagging →
Related concepts
- random forestused by · A random forest is an ensemble model that combines predictions from multiple decision trees built with randomized data samples, feature selections, or both.
- bootstrap samplinguses · Bootstrap sampling is a resampling technique that repeatedly draws observations with replacement from observed data to estimate a statistic’s sampling behavior or variability.
- modeuses · The statistical mode is the value or category occurring most frequently in a set of observations or outcomes.
- random subspacesincludes · Random subspaces is an ensemble technique that trains each model on a randomly selected subset of input features.
- data aggregationuses · Data aggregation is the combination of multiple values, records, observations, or results into a consolidated or summarized output.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.