Random subspaces is an averaging-based ensemble technique: each model trains on a randomly selected subset of features, then model predictions are combined to improve reliability and reduce variance. section 5.1.1
Random subspaces
Random subspaces is an ensemble sampling technique that trains each model using a randomly selected subset of features.
How Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI covers it
- How deep it goes
- Defines feature sampling and configuration parameters, but does not provide a dedicated worked implementation.
- Examples
- Contrasts feature-only sampling with training-example sampling and combined sampling in a strategy table.
- What it assumes
- Assumes familiarity with single learning algorithms, decision trees, and ensemble prediction through averaging.
- What you can do after
- Operate/configure BaggingClassifier or BaggingRegressor using feature subsampling and full training-example samples.
- How it connects
- Connects random subspaces with averaging, bagging, pasting, random patches, random forests, and variance reduction.
Also explained in
-
Outlier Detection in PythonRandom subspace selection is a technique for constructing feature subsets by choosing dimensions randomly.
-
Ensemble Methods for Machine LearningRandom subspaces is an ensemble technique that trains component models using randomly selected subsets of input features.
Compare how 3 books explain random subspaces →
Related concepts
- random forestcontrasts with · A random forest is an ensemble model that combines predictions from multiple decision trees built with randomized data samples, feature selections, or both.
- baggingkind of · Bagging, or bootstrap aggregation, is an ensemble technique that trains multiple models on bootstrap samples of data and combines their predictions.
- feature baggingused by · Feature bagging is an ensemble technique that trains multiple detectors on randomly selected feature subsets and combines their outputs.
- subspaceproduces · A subspace is a subset of a vector space that contains the zero vector and is closed under vector addition and scalar multiplication.
- BaggingClassifierimplemented by · BaggingClassifier is a scikit-learn ensemble estimator that trains base estimators on bootstrap samples and aggregates their predictions.
- decision treeuses · A decision tree is a branching structure that represents sequential tests or choices, with paths leading to outcomes, predictions, or actions.
This explainer was generated by Manning's knowledge graph from the text of Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI. The section links point to the parts of the book it draws on.