Parallel ensemble method

As explained in Ensemble Methods for Machine Learning

A parallel ensemble is an ensemble whose component base models are trained independently of one another.

A parallel ensemble trains multiple base models independently, then combines their predictions into one answer. Like a diagnostician consulting independent residents, it benefits from diverse opinions and aggregation. section 2.1 section 1.5

A parallel ensemble typically follows this pattern:

• Choose a base-learning algorithm or a collection of different algorithms. section 1.5
• Train each base estimator independently, allowing training to occur in parallel. section 2.0
• Create diversity by randomly subsampling training examples, features, or both, or by changing the learning algorithm. section 2.0 section 7.2
• Aggregate the individual predictions into a final classification or regression prediction. section 2.1 section 7.2

Homogeneous parallel ensembles use the same learning algorithm for every base model; bagging, random forests, and Extra Trees are examples. Heterogeneous ensembles use different algorithms, as in stacking. Parallel ensembles often use strong learners, unlike sequential ensembles, which commonly build strong models from dependent weak learners. section 1.5 section 7.2

Regression with random forests

For bike-rental prediction, RandomForestRegressor and ExtraTreesRegressor can be tuned by exhaustive grid search and cross-validation. Useful hyperparameters include ensemble size and maximum base-estimator depth; candidate loss functions include squared error and Poisson. section 7.4.3

Figure 2.1 Dr. Randy Forrest’s diagnostic process is an analogy of a parallel ensemble method.

Use the diagnostic analogy to visualize independent base estimators contributing opinions that are combined into one prediction. section 2.1

How Ensemble Methods for Machine Learning covers it

How deep it goes

Explains training independence, diversity generation, aggregation, taxonomy, and practical tuning without deriving ensemble theory.

Examples

Uses a diagnostic team analogy, random forests and Extra Trees, AutoMPG regression, and bike-rental prediction.

What it assumes

Assumes basic machine-learning models and introduces ensemble diversity, base estimators, and aggregation through intuition and examples.

What you can do after

Apply parallel ensembles with scikit-learn regressors, feature or data sampling, aggregation, grid search, and cross-validation.

How it connects

Connects ensemble diversity and model aggregation with bagging, random forests, Extra Trees, stacking, and sequential ensembles.

liveBookmanning.com

Read Ensemble Methods for Machine Learning on liveBook Get the book at manning.com

Related concepts

• sequential ensemble methodcontrasts with · A sequential ensemble method trains base learners in successive stages, with later learners depending on or correcting the outputs of earlier learners.
• ensemblekind of · An ensemble is a technique that combines or considers outputs from multiple models or component methods collectively to produce or assess a joint result.
• baggingincludes · Bagging, or bootstrap aggregation, is an ensemble technique that trains multiple models on bootstrap samples of data and combines their predictions.
• strong learneruses · A strong learner is a predictive model with substantially improved task performance, often formed by combining multiple weak learners.
• base learnerhas part · A base learner is an individual predictive model whose output contributes to an ensemble or higher-level estimator.
• homogeneous ensembleincludes · A homogeneous ensemble is an ensemble whose base learners are instances of the same model or learning algorithm.

This explainer was generated by Manning's knowledge graph from the text of Ensemble Methods for Machine Learning. The section links point to the parts of the book it draws on.