Bagging

Q: How deeply does Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI cover bagging?

Explains sampling variants, aggregation rules, parameter settings, trade-offs, and a scikit-learn workflow without mathematical derivation.

As explained in Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI

A collection of models trained on resampled data whose predictions are aggregated to reduce variance.

Bagging improves predictive reliability by training multiple models on bootstrapped versions of the training data and aggregating their predictions. It is an averaging ensemble: regression predictions are averaged, while classification predictions use majority voting. The main practical goal is to reduce prediction variance without greatly increasing bias. section 5.1.1 section 5.6

Use bagging as one member of a broader family of sampling-based averaging strategies:

Pasting samples training examples without replacement. section 5.1.1
Bagging samples training examples with replacement, also called bootstrapping. section 5.1.1 section 5.6
Random subspaces sample features, while random patches sample both training examples and features. section 5.1.1

A bagged tree classifier

A practical classifier can use scikit-learn's BaggingClassifier with a DecisionTreeClassifier base estimator, 300 estimators, bootstrapping enabled, and all samples and features available to each model. A preprocessing pipeline can one-hot encode categorical columns and pass through numeric columns. The recorded result is an accuracy of 0.809 with a 0.004 variation, requiring 37.93 seconds to fit and 0.83 seconds to predict.

Bagging can improve results by making predictions less sensitive to the quirks of one training sample. The trade-off is computational cost and possible bias when sampling excludes important parts of the data distribution. Random forests extend the idea by combining bootstrapped samples with feature subsampling. section 5.1.1 section 5.6

How Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI covers it

Cover of Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI

liveBook manning.com

Read Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI on liveBook Get the book at manning.com

Also explained in

liveBook manning.com

Ensemble Methods for Machine LearningBagging is an ensemble technique that trains multiple models independently and combines their predictions.
liveBook manning.com

Grokking Machine LearningAn ensemble technique that trains successive learners on random bootstrap subsets of data and aggregates their predictions.
liveBook manning.com

Machine Learning Algorithms in DepthBagging is an ensemble technique that trains models on bootstrap-resampled data and aggregates their predictions to reduce variance.

Compare how 4 books explain bagging →

Related concepts

random forestused by · A random forest is an ensemble model that combines predictions from multiple decision trees built with randomized data samples, feature selections, or both.
bootstrap samplinguses · Bootstrap sampling is a resampling technique that repeatedly draws observations with replacement from observed data to estimate a statistic’s sampling behavior or variability.
modeuses · The statistical mode is the value or category occurring most frequently in a set of observations or outcomes.
random subspacesincludes · Random subspaces is an ensemble technique that trains each model on a randomly selected subset of input features.
data aggregationuses · Data aggregation is the combination of multiple values, records, observations, or results into a consolidated or summarized output.

This explainer was generated by Manning's knowledge graph from the text of Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI. The section links point to the parts of the book it draws on.