Bagging

As explained in Grokking Machine Learning

An ensemble technique that trains successive learners on random bootstrap subsets of data and aggregates their predictions.

Bagging, short for bootstrap aggregating, builds several learners on random samples drawn from a dataset with replacement, then combines their predictions. Classification uses a vote; regression uses an average. section 12.1 section 12.7

A bagging procedure:

• Draw random subsets from the dataset with replacement; some points may appear in multiple subsets, while others may be left out. section 12.1
• Train a separate learner on each subset, then combine their predictions by voting for classification or averaging for regression. section 12.1

Spam and ham emails

A random forest can combine decision trees trained on samples of emails described by counts of “lottery” and “sale.” In a manual illustration, three trees vote; a Scikit-Learn example fits a forest of five depth-1 trees. section 12.2

Figure 12.1 Ensemble methods consist of joining several weak learners to build a strong learner.

The figure illustrates how several weak learners combine into a strong learner. section 12.1

How Grokking Machine Learning covers it

How deep it goes

Defines bootstrap sampling and prediction aggregation, then works through a random-forest example without deriving theoretical guarantees.

Examples

Uses a friends’ voting analogy and a spam-email dataset with lottery and sale word counts.

What it assumes

Assumes familiarity with decision trees and introduces learner as another term for a machine-learning model.

What you can do after

Apply bagging via Scikit-Learn’s RandomForestClassifier to fit a forest and score its predictions.

How it connects

Connects bagging with ensemble learning, random forests, decision trees, boosting, classification, and regression.

liveBookmanning.com

Read Grokking Machine Learning on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Ensemble Methods for Machine LearningBagging is an ensemble technique that trains multiple models independently and combines their predictions.
• 

  liveBookmanning.com
  Machine Learning for Tabular Data: XGBoost, Deep Learning, and AIA collection of models trained on resampled data whose predictions are aggregated to reduce variance.
• 

  liveBookmanning.com
  Machine Learning Algorithms in DepthBagging is an ensemble technique that trains models on bootstrap-resampled data and aggregates their predictions to reduce variance.

Compare how 4 books explain bagging →

Related concepts

• random forestused by · A random forest is an ensemble model that combines predictions from multiple decision trees built with randomized data samples, feature selections, or both.
• bootstrap samplinguses · Bootstrap sampling is a resampling technique that repeatedly draws observations with replacement from observed data to estimate a statistic’s sampling behavior or variability.
• modeuses · The statistical mode is the value or category occurring most frequently in a set of observations or outcomes.
• random subspacesincludes · Random subspaces is an ensemble technique that trains each model on a randomly selected subset of input features.
• data aggregationuses · Data aggregation is the combination of multiple values, records, observations, or results into a consolidated or summarized output.

This explainer was generated by Manning's knowledge graph from the text of Grokking Machine Learning. The section links point to the parts of the book it draws on.