Classification score

As explained in Grokking Machine Learning

A signed numerical output of a linear classifier whose sign indicates the predicted class and whose magnitude indicates distance from the decision boundary.

Can word counts help decide whether a sentence sounds happy or sad? A linear classifier first gives the sentence a numerical score, then uses its sign to choose a class. For a simple word-count classifier, give each appearance of “aack” one point and each appearance of “beep” minus one point. Add the points: a positive or zero score predicts happy; a negative score predicts sad. section 5.1

Score a sentence

“Aack beep aack” scores 1 − 1 + 1 = 1, so it is predicted happy. “Beep beep” scores −2, so it is predicted sad. Equal counts score zero and, by this classifier’s chosen rule, predict happy. Try scoring a new sentence before checking which side of the boundary its word counts fall on. section 5.1

More generally, multiply each feature by its weight, add the products, and add a bias: score = w₁x₁ + w₂x₂ + … + wₙxₙ + b. The features might be word counts; the bias shifts the boundary and determines the score when every count is zero. Points scoring zero lie on the decision boundary. Negating every weight and the bias keeps the boundary but swaps its positive and negative sides. section 5.1 section 5.6

Figure 5.3 The classifier is the diagonal line that splits the happy and the sad points. The equation of this line is x_aack = x_beep (or equivalently, x_aack – x_beep = 0), because the line corresponds to all the points where the horizontal and the vertical coordinates are equal. The happy zone is the zone in which the number of appearances of aack is greater than or equal to the number of appearances of beep, and the sad zone is the zone in which the number of appearances of aack is less than that of beep.

The word-count plot makes the score’s sign visible: equal counts form the zero-score line, with positive and negative predictions on opposite sides. section 5.1

A score carries more information than the final category. If the true label is sad, scores of 1 and 100 are both mistakes, but the latter signals a much larger error for that classifier. Counting mistakes alone misses this difference. Geometrically, a misclassified point farther across the boundary is a worse error; perpendicular distance can be used to measure it. Do not treat an unadjusted score magnitude as that perpendicular distance: changing the weights can change score sizes without changing the boundary. section 5.2 section 5.1

Keep the raw score distinct from an activated prediction. A perceptron applies a step function to turn the score into a class. In the coded logistic classifier, `score(weights, bias, features)` computes the weighted sum, while `prediction(...)` applies the sigmoid function to that sum. For multiple classes, separate classifier scores can instead be passed through softmax to produce probabilities. section 5.1 section 6.3 section 6.5

How Grokking Machine Learning covers it

How deep it goes

Works from word-count arithmetic to weighted sums, boundary geometry and error severity, without deriving a distance formula.

Examples

Scores happy and sad sentences by word counts and compares differently sized mistakes on a sad example.

What it assumes

Starts with counting words and adding signed points before introducing weighted features, bias and geometric boundaries.

What you can do after

Apply it to calculate a linear classifier’s score, make a binary prediction and recognize when mistake counts hide error severity.

How it connects

Connects perceptrons, decision boundaries, bias, step functions, sigmoid functions, softmax and classifier error.

liveBookmanning.com

Read Grokking Machine Learning on liveBook Get the book at manning.com

Related concepts

• perceptron error functionused by · A perceptron loss function that assigns zero to correctly classified instances and the absolute score to misclassified instances.
• interceptequivalent to · An intercept is an additive constant in a model that shifts its output independently of its input features.
• inner productuses · An inner product is a scalar-valued operation on two vectors that combines their corresponding components, summing their products and, for complex vectors, using conjugation.
• weightequivalent to · A weight is a numerical quantity assigned to an element to represent its contribution, influence, importance, strength, or resource amount in a calculation or system.
• softmaxused by · Softmax is a function that exponentiates a vector of scores and normalizes the results into nonnegative values that sum to one.
• sentiment classifierproduced by · A sentiment classifier is a machine-learning model that assigns sentiment categories or scores to input content, especially text.

This explainer was generated by Manning's knowledge graph from the text of Grokking Machine Learning. The section links point to the parts of the book it draws on.