liveBook · Manning

Linear stacking

As explained in Ensemble Methods for Machine Learning

Linear stacking is stacking in which the second-level estimator is a linear model.

Linear stacking combines diverse level-1 estimators by feeding their predictions or probabilities into a linear level-2 model. Linear regression and logistic regression are common choices because they train quickly and can support exploratory analysis. section 3.3.1

Build a linear stacking model in three stages:

  • Train the individual base estimators. section 3.3.1
  • Generate meta-features from their predictions or prediction probabilities. section 3.3.1

Two-moons classification

A stack can combine six diverse classifiers, including decision trees, support vector machines, Gaussian processes, nearest neighbors, random forests, and Gaussian naive Bayes. Logistic regression can then learn how to combine their outputs. section 3.3.1

How Ensemble Methods for Machine Learning covers it

How deep it goes
Explains the three-stage fitting and prediction workflow, but does not detail safeguards against stacking overfit.
Examples
Uses two-moons classification, six heterogeneous classifiers, prediction probabilities, and logistic regression as the meta-estimator.
What it assumes
Assumes familiarity with supervised estimators, predictions, probabilities, and training data while defining meta-features operationally.
What you can do after
Build it from scratch with fitting and prediction procedures, or operate it through scikit-learn stacking estimators.
How it connects
Connects base estimators, meta-features, ensemble learning, logistic regression, prediction probabilities, and nonlinear stacking.
Cover of Ensemble Methods for Machine Learning
liveBookmanning.com
Read Ensemble Methods for Machine Learning on liveBook Get the book at manning.com

This explainer was generated by Manning's knowledge graph from the text of Ensemble Methods for Machine Learning. The section links point to the parts of the book it draws on.