A classification score is a number that summarizes how strongly a classifier supports one of two classes. Start with a tiny sentiment task: count two words in a sentence, multiply each count by its word weight, and add the results. A nonnegative score can mean happy; a negative score can mean sad. section 5.2 section 5.1
For a perceptron, the score combines every feature with its weight, then adds a bias: weight × feature for each feature, all summed, plus bias. The score of zero is the decision boundary. Scores on opposite sides of zero fall into positive and negative zones, which appear as regions separated by a line. section 5.27 section 5.7
A score with a bias
Suppose crack and doink each have weight 1, while the bias is −3.5. A sentence containing crack once and doink once has score 1 + 1 − 3.5 = −1.5, so it is classified as sad. The bias plays the same role as a threshold: it sets how much positive evidence is needed before the score reaches zero. section 5.4 section 5.5
A score is more informative than a final yes/no label. Two classifiers can make the same wrong prediction, yet the one with a much larger score may be making a more serious mistake because the point lies farther into the wrong zone. Perceptron error uses the absolute score for a misclassified point and zero for a correct one. section 5.12 section 5.19
The same weighted sum also underlies logistic classification. Instead of applying a step function directly, logistic regression applies the sigmoid function to the score. This turns negative and positive scores into values between 0 and 1: zero becomes 0.5, large negative scores approach 0, and large positive scores approach 1. section 6.2 section 6.11
Table 5.4 Our dataset of four sentences with their labels. For each of the two classifiers, we have the score and the prediction.
|
Sentence (xaack, xbeep)
|
Label
y
|
Classifier 1 score
1xaack + 2xbeep – 4
|
Classifier 1 prediction
step(1xaack + 2xbeep – 4)
|
Classifier 1 error
|
Classifier 2 score
–xaack + xbeep
|
Classifier 2 prediction
step(–xaack + xbeep)
|
Classifier 2 error
|
|
(1,0)
|
Sad (0)
|
–3
|
0 (correct)
|
0
|
–1
|
0 (correct)
|
0
|
|
(0,1)
|
Happy (1)
|
-2
|
0 (incorrect)
|
2
|
1
|
1 (correct)
|
0
|
|
(1,3)
|
Happy (1)
|
3
|
1 (correct)
|
0
|
2
|
1 (correct)
|
0
|
|
(3,2)
|
Sad (0)
|
3
|
1 (incorrect)
|
3
|
–1
|
0 (correct)
|
0
|
|
Mean perceptron error
|
|
1.25
|
|
0
|
|||
Compare two classifiers row by row: each weighted score becomes a prediction, and misclassified rows contribute score-based error. section 5.12

