Hinge loss

As explained in Math and Architectures of Deep Learning

A classification loss that is zero once a margin criterion is satisfied and increases only when incorrect classes score too close to or above the correct class.

Hinge loss is a classification loss that stops rewarding improvement once the correct class satisfies a score criterion. Unlike softmax cross-entropy, it does not keep pushing scores toward an unattainable ideal after the correct class is sufficiently favored. section 9.1.8 section 9.4

For a training pair with ground-truth class c, class scores y, and margin m, multiclass SVM hinge loss sums the violations from incorrect classes: max(0, yj − yc + m) for each j ≠ c. The max operation creates the hinge: satisfied constraints contribute zero, while violations contribute increasingly larger loss. section 9.1.8

Interpret the score comparison as follows:

• Good output: an incorrect score below the correct score contributes zero without distinguishing among already-good score configurations. section 9.1.8
• Bad output: an incorrect class that scores too close to or above the correct class contributes positive loss, encouraging separation. section 9.1.8
• Margin-based training can require the correct score to exceed each incorrect score by a fixed amount, commonly m = 1. section 9.1.8

Image classification scores

Suppose an image classifier scores cat, dog, airplane, and automobile classes. With cat as the ground-truth class, hinge loss compares the cat score against every incorrect-class score, penalizing only comparisons that fail the required margin. section 9.1.8

Figure 9.5 Hinge loss for good and bad output predictions

Use the good-and-bad prediction figure to visualize the hinge's zero region and its positive-loss region. section 9.1.8

How Math and Architectures of Deep Learning covers it

How deep it goes

Defines the multiclass SVM formula, margin, and zero-versus-positive cases through score comparisons.

Examples

Uses cat, dog, airplane, and automobile scores to illustrate correct-versus-incorrect class comparisons and good or bad outputs.

What it assumes

Assumes class scores, a ground-truth label, and comparisons between correct and incorrect classes.

What you can do after

Evaluate classification scores by identifying margin violations and recognizing when hinge loss becomes zero.

How it connects

Connects multiclass SVM loss with softmax cross-entropy, focal loss, and optimization.

liveBookmanning.com

Read Math and Architectures of Deep Learning on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Machine Learning Algorithms in DepthA hinge loss function penalizes classification examples whose signed margin falls below a specified threshold.

Compare how 2 books explain hinge loss →

Related concepts

• objective functionkind of · An objective function assigns values to candidate solutions, predictions, or parameter settings according to a criterion to be minimized or maximized.
• cross entropy losscontrasts with · Cross-entropy loss measures the expected negative log probability that a predicted distribution assigns to outcomes drawn from a target distribution.
• marginhas part · A classification margin measures separation or confidence, expressed as distance from a decision boundary or as the correct class's score advantage over competing classes.

This explainer was generated by Manning's knowledge graph from the text of Math and Architectures of Deep Learning. The section links point to the parts of the book it draws on.