liveBook · Manning

Diversity

As explained in Deep Learning with R, Third Edition

Diversity is the degree to which models in an ensemble make different errors or rely on different aspects of the data.

Diversity measures whether ensemble models make different errors or rely on different aspects of the data. Different biases can cancel, improving robustness and accuracy. section 18.1.2

Use diversity as a practical guide when building ensembles:

  • Combine models that are individually strong but use different architectures or machine learning approaches. section 18.1.2
  • Avoid expecting large gains from repeatedly training the same network with different random initializations; the resulting models usually lack diversity. section 18.1.2
  • Consider combining tree-based methods, such as random forests or gradient-boosted trees, with deep neural networks. section 18.1.2

Weighted ensemble predictions

Four validation predictions can be averaged equally, or assigned different weights when some models perform better than others. Validation data can guide the weights.

How Deep Learning with R, Third Edition covers it

How deep it goes
Explains the bias-cancellation rationale and practical model-selection rules, but does not develop a formal diversity metric.
Examples
Uses equal and weighted prediction averages plus a tree-based and neural-network ensemble as practical examples.
What you can do after
Apply ensemble learning by selecting varied models and combining their predictions with equal or validation-optimized weights.
How it connects
Links ensemble learning with model bias, weighted averages, random forests, gradient boosting, and deep neural networks.
Cover of Deep Learning with R, Third Edition
liveBookmanning.com
Read Deep Learning with R, Third Edition on liveBook Get the book at manning.com

Also explained in

Compare how 7 books explain diversity →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Deep Learning with R, Third Edition. The section links point to the parts of the book it draws on.