Perceptron algorithm

As explained in Grokking Machine Learning

A training algorithm that adjusts a perceptron's parameters to improve its predictions on labeled examples.

How can a model decide whether a sentence is happy or sad, or whether an email is spam? A perceptron learns a line that separates two labeled classes. It combines feature values with weights and a bias, then predicts one class when the score is at least zero and the other when it is negative. section 5.0 section 5.6

The perceptron algorithm improves its classifier through a repeating procedure:

• Start with a random line, represented by weights and a bias. section 5.3
• Choose a labeled point and calculate its prediction. section 5.4
• Leave the classifier unchanged when the point is correct; otherwise, move the line slightly toward that point. section 5.3
• Repeat the update, often one randomly selected point at a time, while tracking mean perceptron error. section 5.3 section 5.4

Happy or sad aliens

Use two features—how often an alien says “aack” and “beep”—with labels 1 for happy and 0 for sad. A from-scratch Python implementation computes score = features · weights + bias, applies a step function, updates misclassified examples with a learning rate, and plots the resulting line and error. On the supplied dataset, the learned classifier classifies every point correctly. section 5.4

Figure 5.20 Case 1 (left): A point that is correctly classified tells the line to stay where it is. Case 2 (right): A point that is misclassified tells the line to move closer toward it.

Use the geometric figure to see the key rule: correct points leave the line still, while misclassified points move it closer. section 5.3

A perceptron works best when classes can be separated by a line or hyperplane; with inseparable classes, it searches for the best separation it can find rather than a perfect one. Try building models for patient diagnosis, AND, and OR, then show why XOR cannot be modeled by one perceptron. section 5.6 section 5.7

How Grokking Machine Learning covers it

How deep it goes

Explains geometric and algebraic updates, error measurement, stochastic training, and implementation, while stopping short of multilayer networks.

Examples

Uses happy and sad alien sentences, a patient diagnosis dataset, classifier plots, and AND, OR, and XOR gate exercises.

What it assumes

Assumes basic feature-and-label datasets while developing classification, scores, predictions, error, and updates through concrete examples.

What you can do after

Builds a perceptron from scratch in Python, evaluates its error, and applies a Turi Create classification workflow.

How it connects

Connects classification with linear regression, gradient descent, stochastic gradient descent, neural networks, sentiment analysis, and logical gates.

liveBookmanning.com

Read Grokking Machine Learning on liveBook Get the book at manning.com

Related concepts

• stochastic gradient descentuses · Stochastic gradient descent is an iterative optimization method that updates parameters using gradients estimated from individual examples or randomly selected small batches.
• machine learningkind of · Machine learning is a field of computational methods in which systems learn patterns or relationships from data to make predictions, decisions, or other outputs.
• batch gradient descentalternative to · Batch gradient descent is an optimization method that computes each parameter update using the entire training dataset.
• learning rateuses · A learning rate is a step-size parameter that controls the magnitude of updates made by an iterative learning or optimization process.
• perceptron error functionmeasures · A perceptron loss function that assigns zero to correctly classified instances and the absolute score to misclassified instances.
• lineuses · A line is a one-dimensional straight geometric set extending indefinitely in both directions, commonly represented by a linear equation.

This explainer was generated by Manning's knowledge graph from the text of Grokking Machine Learning. The section links point to the parts of the book it draws on.