Random subspaces

As explained in Ensemble Methods for Machine Learning

Random subspaces is an ensemble technique that trains component models using randomly selected subsets of input features.

Random subspaces builds an ensemble by giving each component model a randomly selected subset of features. Like asking several analysts to inspect different evidence, this creates diverse predictions that can be aggregated. Unlike random forests, which sample features inside randomized tree learning, random subspaces selects features once before training each base estimator. section 2.4.2 section 2.0

A practical workflow is:

• Choose a base learner and the number of component estimators. section 2.4.2
• For each estimator, randomly select a feature subset, with or without replacement. section 2.4.2
• Train the learner using its selected features, then combine the component predictions. section 2.4.2 section 2.6
• Use smaller feature subsets when high dimensionality makes individual models expensive, such as with image or text data. section 2.4.2

Scikit-learn configuration

A BaggingClassifier with SVC as its base estimator can implement random subspaces by setting max_features below all features while leaving training-example sampling disabled: max_samples=1.0, bootstrap=False, max_features=0.5, and bootstrap_features=True. Sampling both examples and features instead produces random patches. section 2.4.2

Figure 2.9 Bagging compared to random subspaces and random patches. The unshaded rows and columns represent training examples and features, respectively, that have been left out.

Compare which training examples and feature columns each ensemble member receives; omitted columns make random-subspace sampling visually distinct from bagging and random patches. section 2.4.2

How Ensemble Methods for Machine Learning covers it

How deep it goes

Defines feature-only sampling, parameterizes it, contrasts it with related ensembles, and stops short of deriving aggregation mathematics.

Examples

Uses SVC, BaggingClassifier settings, and high-dimensional image or text data as concrete examples.

What it assumes

Assumes familiarity with ensemble learning, base estimators, features, and prediction aggregation.

What you can do after

Apply it via scikit-learn's BaggingClassifier with feature sampling and a chosen base learner.

How it connects

Connects bagging, random patches, random forests, ensemble diversity, model aggregation, and unstable estimators.

liveBookmanning.com

Read Ensemble Methods for Machine Learning on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Outlier Detection in PythonRandom subspace selection is a technique for constructing feature subsets by choosing dimensions randomly.
• 

  liveBookmanning.com
  Machine Learning for Tabular Data: XGBoost, Deep Learning, and AIRandom subspaces is an ensemble sampling technique that trains each model using a randomly selected subset of features.

Compare how 3 books explain random subspaces →

Related concepts

• random forestcontrasts with · A random forest is an ensemble model that combines predictions from multiple decision trees built with randomized data samples, feature selections, or both.
• baggingkind of · Bagging, or bootstrap aggregation, is an ensemble technique that trains multiple models on bootstrap samples of data and combines their predictions.
• feature baggingused by · Feature bagging is an ensemble technique that trains multiple detectors on randomly selected feature subsets and combines their outputs.
• subspaceproduces · A subspace is a subset of a vector space that contains the zero vector and is closed under vector addition and scalar multiplication.
• BaggingClassifierimplemented by · BaggingClassifier is a scikit-learn ensemble estimator that trains base estimators on bootstrap samples and aggregates their predictions.
• decision treeuses · A decision tree is a branching structure that represents sequential tests or choices, with paths leading to outcomes, predictions, or actions.

This explainer was generated by Manning's knowledge graph from the text of Ensemble Methods for Machine Learning. The section links point to the parts of the book it draws on.