liveBook · Manning

BaggingRegressor

As explained in Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI

BaggingRegressor is a scikit-learn ensemble estimator that fits base regressors on bootstrap samples and aggregates their predictions.

BaggingRegressor is a scikit-learn ensemble estimator for regression. It trains selected base estimators on sampled training data and combines their predictions, reducing the impact of individual overfit models. section 5.1.1

The estimator supports different sampling strategies through bootstrap, max_samples, and max_features. Its estimator parameter selects the model used in the ensemble; decision trees are the default. section 5.1.1

Use the parameters to choose how diversity enters the ensemble:

  • Bagging samples training examples with replacement. section 5.1.1
  • Pasting samples training examples without replacement. section 5.1.1
  • Random subspaces sample features without replacement. section 5.1.1
  • Random patches sample both training examples and features. section 5.1.1

Regression and stability selection

For a regression problem, replace the classification ensemble with BaggingRegressor and use Lasso as the base estimator. Repeated bootstrap fits can then reveal how consistently features receive relevant coefficients. section 6.2.1

Table 5.1 Bagging and sampling strategies (view table figure)

Averaging strategy

What happens to data

Parameters for BaggingClassifier/BaggingRegressor

Pasting

Training examples are sampled without replacement

bootstrap = False

max_samples < 1.0

max_features = 1.0

Bagging

Training examples are sampled with replacement (bootstrapping)

bootstrap = True

max_samples = 1.0

max_features = 1.0

Random subspaces

Features are sampled (without replacement)

bootstrap = False

max_samples = 1.0

max_features < 1.0

Random patches

Training examples and features are sampled without replacement

bootstrap = False

max_samples < 1.0

max_features < 1.0

Compare bootstrap, sample, and feature settings before configuring a BaggingRegressor. section 5.1.1

How Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI covers it

How deep it goes
Defines four sampling strategies and their parameters, but stops short of a dedicated regression workflow.
Examples
Uses sampling-strategy comparisons, a 300-tree bagged classifier, and bootstrap-based Lasso coefficient selection.
What you can do after
Apply it via scikit-learn by selecting a base estimator and tuning bootstrap, max_samples, and max_features.
How it connects
Connects bootstrap sampling with decision trees, random forests, random subspaces, random patches, Lasso regression, and stability selection.
Cover of Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI
liveBookmanning.com
Read Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI on liveBook Get the book at manning.com

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI. The section links point to the parts of the book it draws on.