liveBook · Manning

Random patches

As explained in Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI

Random patches is an ensemble sampling technique that trains models on randomly selected subsets of both observations and features.

Random patches is an averaging-based ensemble technique for reducing prediction variance. Each model trains on a randomly selected subset of training examples and a randomly selected subset of features, both sampled without replacement. section 5.1.1

The practical goal is to create models that differ from one another. Combining their predictions can produce more reliable results than relying on one model, especially when variance is a concern in tabular-data problems. section 5.1.1 section 5.1.2

Random patches differs from nearby averaging strategies by varying two parts of the training data:

  • Pasting samples training examples without replacement but retains all features. section 5.1.1
  • Bagging samples training examples with replacement while retaining all features. section 5.1.1
  • Random subspaces retain all training examples but sample features without replacement. section 5.1.1
  • Random patches sample both training examples and features without replacement. section 5.1.1

Connection to random forests

Random forests use a related combination of sample bootstrapping and feature sampling. Feature candidates are sampled at each tree split, producing diverse tree structures whose classifications are combined by majority voting. section 5.1.2

Table 5.1 Bagging and sampling strategies (view table figure)

Averaging strategy

What happens to data

Parameters for BaggingClassifier/BaggingRegressor

Pasting

Training examples are sampled without replacement

bootstrap = False

max_samples < 1.0

max_features = 1.0

Bagging

Training examples are sampled with replacement (bootstrapping)

bootstrap = True

max_samples = 1.0

max_features = 1.0

Random subspaces

Features are sampled (without replacement)

bootstrap = False

max_samples = 1.0

max_features < 1.0

Random patches

Training examples and features are sampled without replacement

bootstrap = False

max_samples < 1.0

max_features < 1.0

Compare how pasting, bagging, random subspaces, and random patches alter training examples, features, and BaggingClassifier or BaggingRegressor parameters. section 5.1.1

How Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI covers it

How deep it goes
Defines the sampling contrast among four averaging strategies and connects diversity to more reliable predictions.
Examples
Uses a comparison table and random-forest diagrams showing varied data partitions, tree structures, and majority voting.
What it assumes
Assumes familiarity with single learning algorithms and introduces averaging ensembles before random forests.
What you can do after
Evaluate ensemble sampling choices for reducing variance and producing more reliable predictions.
How it connects
Connects averaging, pasting, bagging, random subspaces, random forests, decision trees, and boosting.
Cover of Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI
liveBookmanning.com
Read Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI on liveBook Get the book at manning.com

Also explained in

Compare how 2 books explain random patches →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI. The section links point to the parts of the book it draws on.