Homogeneous ensemble

As explained in Ensemble Methods for Machine Learning

A homogeneous parallel ensemble uses base learners of the same type trained independently with the same base-learning algorithm.

A homogeneous ensemble is a parallel ensemble whose base learners all use the same learning algorithm, such as decision trees. Rather than asking different model types for opinions, it trains independent versions of one model type and combines their predictions. Because the learners are independent, they can be trained in parallel. section 2.0 section 1.5

Figure 2.1 Dr. Randy Forrest’s diagnostic process is an analogy of a parallel ensemble method.

Like several residents independently assessing a diagnosis, base learners contribute separate opinions that are combined into one decision. The central design problems are creating diverse opinions and choosing an aggregation rule. section 2.1

Using one algorithm does not mean producing identical models. Diversity comes from randomization: sample training examples, features, or both differently for each learner. The ensemble then aggregates the resulting predictions. For classification, this can be majority voting; for regression, it is commonly the mean prediction. section 2.0 section 7.2.1

Common homogeneous parallel methods vary mainly in where they introduce randomness:

• Bagging draws bootstrap samples—samples with replacement—from the training data, fits a base estimator to each sample, and aggregates their predictions. section 2.6
• Random forests are bagging ensembles of randomized decision trees, which evaluate a random subset of features when selecting splits. section 7.2.1
• Extra Trees add stronger tree randomization by selecting both split variables and split thresholds randomly; they do not use bootstrap sampling. section 2.6 section 7.2.1
• Pasting samples examples without replacement; random subspaces sample features; random patches sample both examples and features. section 2.6

These methods are especially suited to unstable, high-complexity or nonlinear estimators, where randomized training can yield meaningfully different fitted models. More diversity can reduce variability and smooth predictions, but every fully trained learner adds computational cost even when training is parallelized. section 2.6 section 7.2.1

Diagnosis and fuel-efficiency prediction

For breast-cancer classification, compare bagged decision trees, random forests, and Extra Trees on the Wisconsin Diagnostic Breast Cancer data. For AutoMPG regression, fit random-forest and Extra-Trees regressors to engine attributes and evaluate predictions after centering and rescaling features. section 2.5 section 7.2 section 7.2.1

How Ensemble Methods for Machine Learning covers it

How deep it goes

Defines the ensemble structure and details bagging, randomized-tree variants, sampling, aggregation, and training-cost tradeoffs.

Examples

Uses a diagnostic-team analogy, Wisconsin breast-cancer classification, and AutoMPG fuel-efficiency regression.

What it assumes

Assumes familiarity with base estimators, classification, regression, and training data while reinforcing sampling and aggregation.

What you can do after

Apply it via scikit-learn random-forest and Extra-Trees regressors for prediction and evaluation.

How it connects

Connects homogeneous ensembles to parallel ensembles, bagging, random forests, Extra Trees, bootstrap sampling, and model aggregation.

liveBookmanning.com

Read Ensemble Methods for Machine Learning on liveBook Get the book at manning.com

Related concepts

• baggingincludes · Bagging, or bootstrap aggregation, is an ensemble technique that trains multiple models on bootstrap samples of data and combines their predictions.
• random forestincludes · A random forest is an ensemble model that combines predictions from multiple decision trees built with randomized data samples, feature selections, or both.
• parallel ensemble methodkind of · A parallel ensemble method combines outputs from multiple independently trained base learners that perform the same task.
• extremely randomized treesincludes · Extremely randomized trees are an ensemble of decision trees that randomizes feature selection and candidate split thresholds during construction.
• heterogeneous ensemblecontrasts with · A heterogeneous ensemble is an ensemble whose component learners use different model types or learning algorithms.

This explainer was generated by Manning's knowledge graph from the text of Ensemble Methods for Machine Learning. The section links point to the parts of the book it draws on.