Ensemble diversity is the extent to which constituent models differ in structure, predictions, or errors. Aggregation benefits when models make different enough errors to offset one another; adding more similar models yields diminishing benefit. section 9.2.3 section 9.3
Random forests create tree diversity through two forms of randomness:
- Bootstrap sampling changes the observations used to train each tree. section 9.3
- At each split, a randomly selected subset of features competes, so dominant predictors cannot shape every tree in the same way. section 9.3
Feature restrictions can make individual trees slightly less accurate, yet produce less-correlated errors and more effective aggregation. Diversity, rather than tree count alone, determines whether additional models contribute meaningfully. section 9.2.3 section 9.3
