Distance error function

As explained in Grokking Machine Learning

An error metric for parallel classifier boundaries equal to the sum of the squared weights, which penalizes boundaries that are close together.

In a support vector machine, the distance error penalizes two parallel boundary lines when they sit too close together. For lines with equations w₁x₁ + w₂x₂ + b = 1 and w₁x₁ + w₂x₂ + b = −1, it is w₁² + w₂²: smaller values correspond to lines farther apart. section 11.1

This term works alongside classification error: one rewards wider spacing, while the other penalizes misclassified points. A combined error can balance the two. section 11.1

Figure 11.7 Left: An SVM where the lines are at distance 0.4 apart, with an error of 25. Right: An SVM where the lines are at distance 0.04 apart, with an error of 2500. Notice that in this comparison, the classifier on the left is much better than the one on the right, because the lines are farther apart from each other. This results in a smaller distance error.

Compare how narrower boundary spacing corresponds to a much larger distance error. section 11.1

How Grokking Machine Learning covers it

How deep it goes

Defines the weight-square formula and explains its geometric effect, while deferring the distance calculation details to an exercise.

Examples

Compares boundary pairs 0.4 units apart with error 25 and 0.04 apart with error 2500.

What you can do after

Evaluate SVM boundaries by relating their spacing to the distance-error penalty.

How it connects

Connects support vector machines, regularization, and classification error.

liveBookmanning.com

Read Grokking Machine Learning on liveBook Get the book at manning.com

Related concepts

• regularizationkind of · Regularization is a modeling technique that constrains or penalizes solutions during fitting or optimization to discourage undesirable complexity or behavior and improve generalization or estimation.
• support vector machineused by · A support vector machine is a supervised-learning model that uses support vectors to determine a maximum-margin decision boundary or regression function, optionally using kernels to model nonlinear relationships.

This explainer was generated by Manning's knowledge graph from the text of Grokking Machine Learning. The section links point to the parts of the book it draws on.