Perceptron trick

As explained in Grokking Machine Learning

A perceptron update technique that adjusts weights and bias according to the signed difference between a point's label and prediction.

The perceptron trick makes a small, point-by-point improvement to a classifier. Leave the boundary line unchanged for a correctly classified point; for a misclassified point, move it slightly toward that point. section 5.3

Use the trick within the perceptron training loop:

• Start with a random classifier, then select a point. Keep the classifier as it is if the point is correctly classified; otherwise, move it slightly toward the point. section 5.3
• Repeat the improvement step many times, then use perceptron error to decide when to stop. section 5.3

Try an update

For the model ŷ = step(2x₁ + 3x₂ − 4), examine the point (1, 1) with label 0. Check its prediction and error, then use the trick with learning rate 0.01 to find a new model with smaller error. section 5.7

Figure 5.20 Case 1 (left): A point that is correctly classified tells the line to stay where it is. Case 2 (right): A point that is misclassified tells the line to move closer toward it.

Compare the two cases: a correctly classified point leaves the line in place, while a misclassified point moves it closer. section 5.3

How Grokking Machine Learning covers it

How deep it goes

Explains the pointwise geometric update and identifies an equivalent algebraic view.

Examples

Uses a moving boundary line, a point-update exercise, and COVID diagnosis and logic-gate datasets.

What you can do after

Apply the update to improve a classifier for an individual point and work through labeled datasets.

How it connects

Connects perceptron training to perceptron error, gradient descent, and linear regression.

liveBookmanning.com

Read Grokking Machine Learning on liveBook Get the book at manning.com

Related concepts

• perceptron algorithmused by · The perceptron algorithm is an iterative linear-classification procedure that updates model parameters when labeled examples are misclassified.
• stochastic gradient descentused by · Stochastic gradient descent is an iterative optimization method that updates parameters using gradients estimated from individual examples or randomly selected small batches.
• learning rateconfigures · A learning rate is a step-size parameter that controls the magnitude of updates made by an iterative learning or optimization process.

This explainer was generated by Manning's knowledge graph from the text of Grokking Machine Learning. The section links point to the parts of the book it draws on.