Hinge loss

As explained in Machine Learning Algorithms in Depth

A hinge loss function penalizes classification examples whose signed margin falls below a specified threshold.

Hinge loss measures classification disagreement through the signed margin zᵢ = yᵢ(θ · xᵢ + θ₀). It penalizes examples with insufficient margin and is minimized by stochastic gradient descent. section 5.2

Its optimization behavior follows from its piecewise-linear shape:

• Use zero gradient when the signed margin exceeds 1. section 5.2
• Use a subgradient at margin 1 because the maximum operator is not differentiable there. section 5.2
• Update the classifier parameters for margins at or below 1 through stochastic gradient descent. section 5.2

Figure 5.7 Hinge loss, 0–1 loss, and log-loss functions

Compare hinge loss with 0–1 loss and log-loss across classification margins. section 5.2

How Machine Learning Algorithms in Depth covers it

How deep it goes

Derives signed margins, nondifferentiability, subgradients, and parameter updates, stopping before a separate SVM treatment.

Examples

Uses an iris perceptron implementation and compares hinge loss with 0–1 loss and log-loss.

What it assumes

Assumes labeled examples, linear predictions, signed margins, gradients, and stochastic gradient descent.

What you can do after

Apply it via stochastic gradient descent to understand and reproduce perceptron parameter updates.

How it connects

Links classification, perceptrons, stochastic gradient descent, subgradients, and support vector machines.

liveBookmanning.com

Read Machine Learning Algorithms in Depth on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Math and Architectures of Deep LearningA classification loss that is zero once a margin criterion is satisfied and increases only when incorrect classes score too close to or above the correct class.

Compare how 2 books explain hinge loss →

Related concepts

• objective functionkind of · An objective function assigns values to candidate solutions, predictions, or parameter settings according to a criterion to be minimized or maximized.
• cross entropy losscontrasts with · Cross-entropy loss measures the expected negative log probability that a predicted distribution assigns to outcomes drawn from a target distribution.
• marginhas part · A classification margin measures separation or confidence, expressed as distance from a decision boundary or as the correct class's score advantage over competing classes.

This explainer was generated by Manning's knowledge graph from the text of Machine Learning Algorithms in Depth. The section links point to the parts of the book it draws on.