Perceptron

As explained in Math and Architectures of Deep Learning

A simple linear threshold unit that computes a weighted sum of inputs.

A perceptron is a binary linear classifier: it takes a weighted sum of input features, adds a bias, and applies a step function. Its output is 0 on one side of a hyperplane and 1 on the other, so the weights set the boundary’s orientation while the bias sets its position. section 7.3.3 section 7.6

For an input vector x, the computation is a step function applied to w·x + b. Geometrically, w·x + b = 0 is the decision boundary; changing w rotates or reorients that boundary, while changing b shifts it. A single perceptron therefore succeeds only when the two classes can be separated by one planar surface. section 7.3.3 section 7.6

Figure 7.8 although the input space is 2D, the perceptron graph is 3D. The decision boundary is indicated by the long diagonal straight line. It maps points on one side of the decision boundary to the Z = 0 plane and the points on the other side to the Z = 1 plane.

View the thresholded output as two flat output regions separated by a line in a two-dimensional input space. section 7.3.3

A perceptron can express simple Boolean rules by choosing weights and bias values:

• AND uses two positive input weights and a threshold that fires only when both binary inputs are 1. section 7.3.4
• OR and NOT likewise correspond to decision boundaries that separate their binary input points. section 7.3.4
• XOR cannot be represented by one perceptron because no single straight boundary separates its positive and negative points. section 7.4.1

Implementing a threshold unit

A PyTorch-style implementation augments each input with a constant 1, joins the bias to the weights, performs matrix multiplication, and applies torch.heaviside. This treats a batch of inputs as a fully connected layer with a threshold activation. section 7.3.3

Multiple perceptrons overcome the single-boundary limit. Their outputs can be combined in a multilayer perceptron (MLP): for example, several threshold units can enclose a rectangular decision region, and layered units can model XOR. More generally, a single-hidden-layer MLP can approximate continuous functions, although it may require impractically many perceptrons. section 7.4.1 section 7.5.3 section 7.5.4

Hard step functions are expressive but not differentiable at zero, which prevents gradient-based optimization of weights. Training therefore uses differentiable step-like functions such as sigmoid; in layered networks, each unit’s weighted sums are efficiently written as matrix-vector multiplication. section 8.1 section 8.3.1

How Math and Architectures of Deep Learning covers it

How deep it goes

Derives threshold geometry, logic-gate constructions, multilayer combinations, and a matrix-based PyTorch implementation; defers parameter training to gradient-based methods.

Examples

Uses AND, OR, NOT, XOR, planar and polygonal decision regions, plus PyTorch threshold-unit code.

What it assumes

Introduces classification through labeled input points and develops the weighted-sum threshold unit before training is addressed.

What you can do after

Evaluate/understand when one thresholded linear unit can classify data and why multilayer networks are needed for nonlinearly separable functions.

How it connects

Connects to Heaviside and sigmoid functions, hyperplanes, linear classifiers, MLPs, XOR, matrix-vector multiplication, loss, and gradient-based training.

liveBookmanning.com

Read Math and Architectures of Deep Learning on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Grokking Machine LearningA processing unit or model that computes a weighted combination of inputs, adds a bias, and applies an activation function.
• 

  liveBookmanning.com
  Natural Language Processing in Action, Second EditionA perceptron is a single-layer computational model that computes a weighted sum of inputs and applies a thresholding activation to make a decision.
• 

  liveBookmanning.com
  Grokking AI Algorithms, Second EditionA perceptron is a computational unit and binary linear classifier that combines weighted inputs with a bias and applies an activation rule.
• 

  liveBookmanning.com
  Machine Learning Algorithms in DepthA linear binary classifier that predicts a class from the sign of a weighted sum of input features.

Compare how 5 books explain perceptron →

Related concepts

• classifierkind of · A classifier is a predictive computational model that assigns inputs to one or more discrete classes or categories.
• neuronkind of · A neuron is a computational unit in a neural network that transforms inputs, typically using learned parameters and an activation function, to produce an output.
• step functionuses · A step function is a piecewise-constant function that changes abruptly between discrete output values when its input crosses one or more specified thresholds.
• language modelpart of · A language model is a computational model that represents patterns in linguistic tokens to estimate sequence probabilities and predict or generate language from context.
• logical AND operatorimplements · A logical AND operator is a Boolean conjunction that produces true exactly when all operands have true truth values.
• logistic regressioncontrasts with · Logistic regression is a statistical classification model that estimates categorical outcome probabilities by applying a logistic or softmax function to a linear combination of input features.

This explainer was generated by Manning's knowledge graph from the text of Math and Architectures of Deep Learning. The section links point to the parts of the book it draws on.