liveBook · Manning

Strength

As explained in Timeless Algorithms, The Foundational Ideas

Tree strength measures the predictive ability of individual trees in a forest.

In a classification forest, strength is the expected margin across observations: how decisively the forest favors the known class over its strongest competitor. section 9.4.2

s=EX, Y[margin(X, Y)]

The equation defines strength s as the population expectation of the forest margin. section 9.4.2

Estimate strength by averaging margins across labeled out-of-bag or test observations. A large positive value signals a consistent voting advantage; a value near zero signals divided votes, and a negative value signals poor classification. section 9.4.2

PE*≤ρ̄(1-s2)s2

The generalization-error bound relates forest error to strength s and average tree correlation ρ̄: greater strength helps, while greater correlation weakens the bound. section 9.4.2

Strength is necessary but not sufficient for useful aggregation: strong, highly correlated trees repeat errors, while diverse but weak trees contribute little reliable information. Random feature selection can reduce correlation without sacrificing too much strength. section 9.4 section 9.4.2

How Timeless Algorithms, The Foundational Ideas covers it

How deep it goes
Defines strength as expected margin, gives an empirical estimate and interpretation, and situates it in a generalization-error bound.
Examples
Uses labeled out-of-bag or test observations as the cases for averaging margins.
What you can do after
Evaluate classification strength by averaging margins over labeled out-of-bag or test observations and interpreting their sign and size.
How it connects
Connects forest margin and generalization error to tree correlation, random feature selection, and ensemble aggregation.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Timeless Algorithms, The Foundational Ideas. The section links point to the parts of the book it draws on.