Meta-learning

As explained in Ensemble Methods for Machine Learning

Meta-learning is a machine-learning approach that learns how to select, combine, or improve other learning procedures.

Meta-learning combines different models by learning how to use their predictions. Instead of hand-designing weights or another fixed rule, it treats base-estimator predictions as metadata and feeds them to a second-level learner. section 3.0 section 3.3

A simple stacking workflow is:

• Train diverse base estimators, using different learning algorithms to create heterogeneous predictions. section 3.5
• Use their predictions as inputs to a meta-estimator, which learns the combination function from data. section 3.3
• Use cross-validation to reduce overfitting, or use a single validation set for the faster blending procedure. section 3.5

Stacking in recommendation

Stacking became prominent in recommendation systems after top teams used it during the Netflix Prize competition. The same approach has also been applied to chemometrics, text classification, and spam filtering. section 3.3

Figure 3.1 Homogeneous ensembles (chapter 2), such as bagging and random forests, use the same learning algorithm to train base estimators, and they achieve ensemble diversity through random sampling. Heterogeneous ensembles (this chapter) use different learning algorithms to achieve ensemble diversity.

Compare fixed weighting with learned prediction combination, and distinguish homogeneous ensembles from heterogeneous ensembles. section 3.0

How Ensemble Methods for Machine Learning covers it

How deep it goes

Explains two-level stacking, deeper layers, cross-validation, blending, and overfitting tradeoffs without detailing implementation code.

Examples

Uses recommendation systems, the Netflix Prize competition, chemometrics, text classification, and spam filtering as application examples.

What it assumes

Assumes familiarity with base estimators, predictions, ensemble methods, validation, and overfitting.

What you can do after

Apply it via stacking or blending to train a learned combination of multiple model predictions.

How it connects

Contrasts weighting methods and connects heterogeneous ensembles with stacking, blending, cross-validation, and meta-estimators.

liveBookmanning.com

Read Ensemble Methods for Machine Learning on liveBook Get the book at manning.com

Related concepts

• machine learningkind of · Machine learning is a field of computational methods in which systems learn patterns or relationships from data to make predictions, decisions, or other outputs.
• heterogeneous ensembleused by · A heterogeneous ensemble is an ensemble whose component learners use different model types or learning algorithms.
• meta-learneruses · A meta-learner is a second-level learning model that combines predictions produced by base estimators to make final predictions.

This explainer was generated by Manning's knowledge graph from the text of Ensemble Methods for Machine Learning. The section links point to the parts of the book it draws on.