A perceptron is a small decision-making model inspired by a biological neuron. Each numerical input has a weight that controls its influence. The perceptron adds the weighted inputs, then applies an activation function to produce an output. For text, the inputs can be word counts or TF-IDF values rather than signals from physical sensors. section 5.1.3 section 5.1.4
The calculation is a weighted sum: z = w₀ + w₁x₁ + … + wₙxₙ. Here w₀ is the intercept, and each other weight changes the model’s sensitivity to one feature. A step activation returns 1 when z exceeds a threshold and 0 otherwise. A logistic activation instead gives a continuous output through a soft threshold. This makes the calculation closely related to logistic regression. section 5.1.4 section 5.1.5
Count words, then make a decision
The phrase “green egg egg ham ham ham spam spam spam spam” becomes a bag-of-words vector with counts 1, 2, 3, and 4. With an intercept of 0.07 and weights 0.12, −0.16, 0.03, and −0.18, the weighted sum is −0.76. A step activation with threshold zero therefore returns 0. These weights are random: the result demonstrates the calculation, not a trained prediction. section 5.1.5
>>> from collections import Counter >>> np.random.seed(451) >>> tokens = "green egg egg ham ham ham spam spam spam spam".split() >>> bow = Counter(tokens) >>> x = pd.Series(bow) >>> x green 1 egg 2 ham 3 spam 4 >>> x1, x2, x3, x4 = x >>> x1, x2, x3, x4 (1, 2, 3, 4) >>> w0 = np.round(.1 * np.random.randn(), 2) >>> w0 0.07 >>> w1, w2, w3, w4 = (.1 * np.random.randn(len(x))).round(2) >>> w1, w2, w3, w4 (0.12, -0.16, 0.03, -0.18) >>> x = np.array([1, x1, x2, x3, x4]) #1 >>> w = np.array([w0, w1, w2, w3, w4]) #2 >>> y = np.sum(w * x) #3 >>> y -0.76
The Python listing constructs the word counts, includes the intercept as an input of 1, and computes the weighted sum with NumPy. section 5.1.5
To learn useful weights, a model needs examples and a loss function that measures prediction error; the calculation above shows only its feed-forward path. One application predicts a birth-certificate sex label from the characters in a first name. The reported single-neuron accuracy is about 80%, not certainty about any individual: names can occur with more than one label, and binary birth-certificate categories oversimplify sex. Keep names in the test set out of training when evaluating such a model. section 5.1.5 section 5.2.2
