Margin
In classification, a margin is a measure of separation between a decision boundary and data points, or of the correct class’s score advantage over competing classes.
Timeless Algorithms, The Foundational Ideas
A boosting margin measures the combined classifier's support for the correct class relative to a competing class.
- How it is used
- Uses boosting margins to explain how an ensemble can improve on new data after reaching perfect training accuracy.
- How deep it goes
- Defines a boosting margin qualitatively and relates it to generalization, without specifying its calculation or a training procedure.
- Examples
- Contrasts random forest vote margins of 0.70 and 0.02 for two correct classifications.
- What you can do after
- Evaluate a classifier’s separation beyond accuracy and distinguish vote-based margins from an SVM’s geometric margin.
Read how Timeless Algorithms, The Foundational Ideas explains margin →
Machine Learning System Design
A measure of the separation between observations and the classifier boundary that determines their predicted classes.
- How it is used
- Uses metric sensitivity to sample noise to assess whether a classifier’s decision boundary is robust.
- How deep it goes
- Explains threshold sensitivity, noise testing, bootstrap distributions, and confidence bounds without deriving classifier geometry.
- Examples
- Demonstrates robustness using precision-recall distributions, bootstrapped datasets, sample sizes, and added normally distributed noise.
- What you can do after
- Diagnose boundary fragility by comparing noisy and unnoised metric distributions and reporting bootstrapped lower confidence bounds.
Machine Learning Algorithms in Depth
A margin is the distance between a classifier's decision boundary and the nearest training examples or class-separating boundaries.
- How it is used
- Uses geometric separation to select among correctly classifying decision boundaries in support vector machines.
- How deep it goes
- Derives a max-min distance objective and its quadratic-program formulation, then stops short of further optimization details.
- Examples
- Uses training examples, margin boundaries, support vectors, and a maximum-margin SVM diagram.
- What you can do after
- Evaluate classifiers by their nearest-example distance and understand why support vectors determine the maximum-margin solution.
Read how Machine Learning Algorithms in Depth explains margin →
Also covered in
What to know first
- convex optimizationConvex optimization is the minimization or maximization of a convex or concave objective over a convex feasible set, a structure that permits global optimality guarantees.
- decision boundaryA decision boundary is a boundary in input or feature space separating regions assigned different classes or decision outcomes by a predictive model.
- classifierA classifier is a predictive model that assigns an input to one or more discrete classes or categories.
Related concepts
- boostingused by · Boosting is an ensemble-learning technique that builds predictive models sequentially and combines them, with later models improving on weaknesses of earlier ones.
- error ratemeasures · Error rate is the proportion of evaluated cases, operations, or outputs that produce an error or fail a specified criterion.
- support vector machineused by · A support vector machine is a supervised-learning model that uses support vectors to define a maximum-margin decision boundary or regression function, with kernels enabling nonlinear relationships.
- high-dimensional datapart of · High-dimensional data consists of observations represented by a large number of features, variables, coordinates, or other dimensions.
Questions about margin
What is margin?
In classification, a margin is a measure of separation between a decision boundary and data points, or of the correct class’s score advantage over competing classes.
Which Manning books explain margin?
Timeless Algorithms, The Foundational Ideas, Machine Learning System Design and Machine Learning Algorithms in Depth.
What should I know before learning margin?
Manning books that teach margin usually introduce convex optimization, decision boundary and classifier first.
What is the difference between margin and vote gap?
In classification, a margin is a measure of separation between a decision boundary and data points, or of the correct class’s score advantage over competing classes. A vote gap is the difference between the vote shares of an observation’s two leading predicted classes.
What is margin part of?
Margin is part of high-dimensional data. High-dimensional data consists of observations represented by a large number of features, variables, coordinates, or other dimensions.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.


