liveBook · Manning

Correlation

As explained in Timeless Algorithms, The Foundational Ideas

Correlation measures the strength and direction of association between quantities.

Correlation describes the strength and direction of association between quantities. In a random forest, it helps explain whether combining many trees produces genuinely different predictions or mostly repeats the same information. section 9.4.2

For classification forests, correlation concerns how trees’ margin behavior moves together across observations. It is high when trees tend to classify and misclassify the same observations; it is lower when their successes and errors occur on different observations. Highly correlated trees offer limited gains from voting, even when each tree is strong. section 9.4.2

Diversity alone is not enough: weak, nearly random trees may make different errors but contribute too little reliable information. A useful forest balances low correlation with strength, defined as the expected margin—the voting advantage of the known class over its strongest competitor. section 9.4.2

PE*≤ρ̄(1-s2)s2

This upper bound relates classification generalization error to forest strength s and average correlation ρ̄. It is a bound, not an exact prediction of error. section 9.4.2

Random feature selection can reduce correlation by preventing the same dominant predictors from shaping every tree. Considering fewer features can also weaken individual trees, so the trade-off matters. Adding trees stabilizes the ensemble’s prediction, but cannot by itself repair weak or highly correlated learners. section 9.9 section 9.4.3

Correlations under changing conditions

A pedestrian-detection network trained mostly on clear daytime images may rely on background or lighting correlations that reduce training loss but fail at night or in severe weather. More daytime images do not fix the missing coverage; performance should be monitored as operating conditions change. section 11.6.3

How Timeless Algorithms, The Foundational Ideas covers it

How deep it goes
Defines correlation through classification margin behavior and relates it to strength in a generalization-error bound.
Examples
Uses correlated tree errors and a pedestrian detector’s reliance on lighting or background to illustrate different consequences of association.
What you can do after
Evaluate and understand how correlation limits ensemble gains and how learned associations may fail under distribution shift.
How it connects
Connects correlation to decision trees, random forests, margins, ensemble learning, and generalization.

Also explained in

Compare how 22 books explain correlation →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Timeless Algorithms, The Foundational Ideas. The section links point to the parts of the book it draws on.