Margin

As explained in Machine Learning Algorithms in Depth

A margin is the distance between a classifier's decision boundary and the nearest training examples or class-separating boundaries.

A margin is the distance between a classifier’s decision boundary and its nearest training examples. A desirable boundary classifies every example correctly while remaining as far away as possible. section 5.3

For each training example, denote its distance from the decision boundary by γᵢ. Maximum-margin classification maximizes the smallest distance, max[minᵢ γᵢ], rather than optimizing any single example’s distance. section 5.3

The SVM formulation connects margin maximization to constrained optimization:

• Minimize the regularized loss by choosing parameters θ with a small L2 norm. section 5.3
• Constrain every training example to remain correctly classified. section 5.3
• As ||θ|| decreases, the distance proportional to 1/||θ|| increases, pushing the margin boundaries apart. section 5.3
• Examples on the final margin boundaries become support vectors; these points suffice to learn the SVM parameters. section 5.3

Figure 5.8 Max margin solution of SVM classifier

Shows the maximum-margin SVM solution, including the decision boundary, margin boundaries, and support vectors. section 5.3

How Machine Learning Algorithms in Depth covers it

How deep it goes

Derives a max-min distance objective and its quadratic-program formulation, then stops short of further optimization details.

Examples

Uses training examples, margin boundaries, support vectors, and a maximum-margin SVM diagram.

What you can do after

Evaluate classifiers by their nearest-example distance and understand why support vectors determine the maximum-margin solution.

How it connects

Links decision boundaries, regularization, quadratic programming, and support vectors.

liveBookmanning.com

Read Machine Learning Algorithms in Depth on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Machine Learning System DesignA measure of the separation between observations and the classifier boundary that determines their predicted classes.

Compare how 2 books explain margin →

Related concepts

• support vector machineused by · A support vector machine is a supervised-learning model that uses support vectors to determine a maximum-margin decision boundary or regression function, optionally using kernels to model nonlinear relationships.
• boostingused by · Boosting is an ensemble-learning technique that builds predictive models sequentially and combines them, with later models improving on weaknesses of earlier ones.
• error ratemeasures · Error rate is the proportion of evaluated operations, cases, or outcomes that produce an error or fail a specified criterion.
• support vectorused by · A support vector is a training example that contributes to determining a support vector machine’s learned prediction function or decision boundary.

This explainer was generated by Manning's knowledge graph from the text of Machine Learning Algorithms in Depth. The section links point to the parts of the book it draws on.