liveBook · Manning

Diversity

As explained in Deep Learning with Python, Third Edition

In ensemble learning, the degree to which models make different errors or rely on different evidence.

Diversity measures how differently ensemble models fail or use evidence. It strengthens ensembles because differently biased models can cancel one another’s errors. section 18.1.2

A simple ensemble

Four classifiers predict validation examples, and their outputs are averaged. This can improve robustness when the models are accurate but make different mistakes. section 18.1.2

preds_a = model_a.predict(x_val)                                     #1
preds_b = model_b.predict(x_val)                                      #1
preds_c = model_c.predict(x_val)                                      #1
preds_d = model_d.predict(x_val)
final_preds = 0.25 * (preds_a + preds_b + preds_c + preds_d)         #2

Average four model predictions equally as a straightforward ensemble baseline. section 18.1.2

Use diversity as a practical selection rule:

  • Prefer models that are individually strong and substantially different, such as different architectures or machine-learning approaches. section 18.1.2
  • Avoid combining copies of one network that differ only by initialization or training-data order; their diversity and added value are usually small. section 18.1.2
  • Try tree-based methods alongside deep neural networks when the problem permits; this combination can work well in practice. section 18.1.2

How Deep Learning with Python, Third Edition covers it

How deep it goes
Explains the practical rationale and selection rules, but does not define a formal diversity metric.
Examples
Uses equal prediction averaging, differently initialized networks, mixed tree-neural ensembles, and the blind-men-and-elephant analogy.
What it assumes
Assumes multiple classifiers and validation predictions, while teaching why shared bias limits ensemble gains.
What you can do after
Evaluate ensemble candidates by balancing individual model quality against architectural and methodological difference.
How it connects
Connects ensemble learning with model bias, weighted averaging, random forests, gradient-boosted trees, and deep neural networks.
Cover of Deep Learning with Python, Third Edition
liveBookmanning.com
Read Deep Learning with Python, Third Edition on liveBook Get the book at manning.com

Also explained in

Compare how 7 books explain diversity →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Deep Learning with Python, Third Edition. The section links point to the parts of the book it draws on.