liveBook · Manning

Forest margin

As explained in Timeless Algorithms, The Foundational Ideas

A forest margin is the difference between a forest's voting support for an observation's correct class and its strongest competing class.

For a labeled observation, a forest margin is the forest’s voting support for the correct class minus its support for the strongest competing class. It measures voting decisiveness, not just whether the winning class is correct; unlike a support vector machine’s geometric margin, it concerns collective votes. section 9.4

s=EX, Y[margin(X, Y)]

Forest strength is the expected margin across the population; labeled out-of-bag or test observations can estimate it by averaging their margins. section 9.4.2

Interpret an average margin as follows:

  • A large positive value indicates a consistent voting advantage for the known class. section 9.4.2
  • A value near zero indicates frequently divided votes; a negative value indicates systematically poor classification. section 9.4.2

Margin-based strength is only part of the generalization story: useful forests need trees that predict better than chance and do not repeatedly make the same errors. Breiman’s error bound relates forest generalization error to strength and average tree correlation. section 9.4 section 9.4.2

How Timeless Algorithms, The Foundational Ideas covers it

How deep it goes
Defines observation-level and expected margins, explains empirical estimation, and relates strength and correlation to an error bound.
Examples
Estimates average margins using labeled out-of-bag or test observations and interprets positive, near-zero, and negative values.
What you can do after
Evaluate a classification forest’s voting decisiveness by interpreting average margins and relating strength to tree correlation.
How it connects
Connects classification margins to random forests, tree strength, correlation, generalization error, and support vector machine margins.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Timeless Algorithms, The Foundational Ideas. The section links point to the parts of the book it draws on.