liveBook · Manning

Margin

As explained in Timeless Algorithms, The Foundational Ideas

A boosting margin measures the combined classifier's support for the correct class relative to a competing class.

A classifier can be correct without winning decisively. In boosting, a margin measures the combined classifier’s support for the correct class relative to a competing class. It adds information about the strength of the separation that training accuracy alone does not capture. section 8.8.3

A boosted classifier combines many individually limited models. AdaBoost can continue increasing margins even after it reaches perfect training accuracy. This offers one explanation for continued improvement on new data despite no further reduction in training error; it is not a guarantee of generalization. section 8.8.3

A related vote margin

In a random forest of 100 trees, suppose the known class is A and the votes are 82 for A, 12 for B, and 6 for C. The margin is 0.82 − 0.12 = 0.70: the vote proportion for the known class minus that of its strongest incorrect competitor. A 51–49 vote also predicts A correctly, but its margin is only 0.02. This is a forest margin, not a geometric distance or a specified boosting calculation. section 9.4.1 section 8.8.3

The distinction matters when comparing models. An SVM’s margin is geometric and enters its training objective; a forest margin describes the voting advantage for a known class. Boosting margins likewise describe combined support rather than the SVM’s distance from a boundary. These notions share an interest in separation beyond mere correctness, but they are not interchangeable. section 8.8.3 section 9.4.1 section 9.7

How Timeless Algorithms, The Foundational Ideas covers it

How deep it goes
Defines a boosting margin qualitatively and relates it to generalization, without specifying its calculation or a training procedure.
Examples
Contrasts random forest vote margins of 0.70 and 0.02 for two correct classifications.
What you can do after
Evaluate a classifier’s separation beyond accuracy and distinguish vote-based margins from an SVM’s geometric margin.
How it connects
Connects boosting margins to AdaBoost, random forests, support vector machines, and generalization.

Also explained in

Compare how 3 books explain margin →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Timeless Algorithms, The Foundational Ideas. The section links point to the parts of the book it draws on.