Perceptron

As explained in Grokking Machine Learning

A processing unit or model that computes a weighted combination of inputs, adds a bias, and applies an activation function.

Can you predict whether a sentence sounds happy or sad from the words it contains? A perceptron is a classifier that gives each input feature a weight, adds a bias, and uses the resulting score to choose between two categories. Unlike regression, which predicts a number, it predicts a category. section 5.1 section 5.6 section 5.0

A first sentiment classifier

Count two words in a sentence: give each appearance of “aack” one point and each “beep” minus one point. A score of zero or more predicts happy; a negative score predicts sad. Plotting the two counts reveals the rule as a diagonal line: on the line, the counts are equal. section 5.1

Figure 5.3 The classifier is the diagonal line that splits the happy and the sad points. The equation of this line is x_aack = x_beep (or equivalently, x_aack – x_beep = 0), because the line corresponds to all the points where the horizontal and the vertical coordinates are equal. The happy zone is the zone in which the number of appearances of aack is greater than or equal to the number of appearances of beep, and the sad zone is the zone in which the number of appearances of aack is less than that of beep.

The diagonal boundary makes the scoring rule visible: sentences with more “aack” fall on one side, and those with more “beep” fall on the other. section 5.1

With more features, the rule is the same: multiply each feature by its weight, add the products and a bias, then apply a step function. In symbols, prediction = step(w₁x₁ + … + wₙxₙ + b). The step function returns 0 for a negative score and 1 otherwise. For two features, the boundary is a line; in higher dimensions, it is a hyperplane. The bias shifts the boundary and determines the prediction when all feature values are zero. section 5.1 section 5.6

How can the boundary improve? The perceptron algorithm starts with a random classifier and repeats a small correction:

• Pick a point. If its category is predicted correctly, leave the classifier alone. section 5.3
• If it is misclassified, adjust the classifier so its boundary moves toward that point. Repeat with other points and measure error to decide when to stop. section 5.3

A line cannot perfectly separate every dataset, so evaluate how many points the classifier gets wrong rather than assuming a perfect split. The same yes-or-no pattern can help frame spam filtering and recommendations. Perceptrons also serve as building blocks of neural networks, where activation functions can differ from the step function used for this classifier. section 5.3 section 5.5 section 10.6

How Grokking Machine Learning covers it

How deep it goes

Works through scores, boundaries, and iterative corrections; also identifies gradient descent as another training route.

Examples

Uses word counts and alien moods to draw a boundary, then names spam filtering and recommendations as applications.

What it assumes

Introduces classification through word counts and plots while comparing the model with linear regression.

What you can do after

Build it from scratch by scoring features, applying a step function, and repeatedly correcting misclassified points.

How it connects

Connects classification to linear regression, decision boundaries, step functions, and neural networks.

liveBookmanning.com

Read Grokking Machine Learning on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Math and Architectures of Deep LearningA simple linear threshold unit that computes a weighted sum of inputs.
• 

  liveBookmanning.com
  Natural Language Processing in Action, Second EditionA perceptron is a single-layer computational model that computes a weighted sum of inputs and applies a thresholding activation to make a decision.
• 

  liveBookmanning.com
  Grokking AI Algorithms, Second EditionA perceptron is a computational unit and binary linear classifier that combines weighted inputs with a bias and applies an activation rule.
• 

  liveBookmanning.com
  Machine Learning Algorithms in DepthA linear binary classifier that predicts a class from the sign of a weighted sum of input features.

Compare how 5 books explain perceptron →

Related concepts

• classifierkind of · A classifier is a predictive computational model that assigns inputs to one or more discrete classes or categories.
• neuronkind of · A neuron is a computational unit in a neural network that transforms inputs, typically using learned parameters and an activation function, to produce an output.
• step functionuses · A step function is a piecewise-constant function that changes abruptly between discrete output values when its input crosses one or more specified thresholds.
• language modelpart of · A language model is a computational model that represents patterns in linguistic tokens to estimate sequence probabilities and predict or generate language from context.
• logical AND operatorimplements · A logical AND operator is a Boolean conjunction that produces true exactly when all operands have true truth values.
• logistic regressioncontrasts with · Logistic regression is a statistical classification model that estimates categorical outcome probabilities by applying a logistic or softmax function to a linear combination of input features.

This explainer was generated by Manning's knowledge graph from the text of Grokking Machine Learning. The section links point to the parts of the book it draws on.