liveBook · Manning

Classifier

As explained in Grokking Machine Learning, Second Edition

The task of assigning an image to one of a set of predefined categories.

A classifier is a machine learning model that chooses a category, also called a state, from a finite set. Give it the features of a new data point—such as an email’s words or an image’s pixels—and it predicts a label such as spam or ham, cat or dog, or yes or no. Unlike regression, which predicts a number, classification makes a category choice. section 2.7 section 2.9

Classifiers learn from labeled data: examples whose correct categories are already known. Features describe each example, while the label is the category to predict. After training, the classifier uses patterns in the features to assign a label to unseen data. section 2.17 section 5.27

A tiny sentiment classifier

Suppose sentences use only the words “aack” and “beep.” Count each word, then predict happy when aack appears at least as often as beep; otherwise predict sad. Each sentence becomes a point whose coordinates are its two word counts. The rule becomes a line that divides a happy region from a sad region. This simple picture builds intuition for a perceptron classifier. section 5.1 section 5.2

A perceptron classifier improves this idea by giving every feature a weight and adding a bias. It calculates a score from the weighted features plus the bias. A nonnegative score produces one class, and a negative score produces the other. The bias plays the same role as moving a cutoff or threshold. section 5.27 section 5.5

A classifier does not need to be perfect to be useful. Real datasets may not be separable by one line, so the aim is to classify points as well as possible while accepting some mistakes. The perceptron algorithm starts with a random classifier, adjusts it repeatedly, and uses error to decide when to stop. section 5.6 section 5.13

Common classification jobs include:

  • Filtering email as spam or not spam, or sorting it into personal, subscriptions, and promotions. section 5.23
  • Recognizing images from pixels, such as handwritten digits or one of ten CIFAR-10 object categories. section 2.9
  • Predicting whether a review is positive or negative, a user will click a link, or two social-media users will interact. section 2.9
  • Predicting whether a patient is sick or healthy, a user will watch a movie, or a voice command was spoken. section 5.0

Simple linear classifiers draw one boundary. When one line cannot separate the categories, several linear classifiers can be combined; this is a basis for neural networks. A neural network can also learn nonlinear boundaries, such as one that separates points near the center from points at the sides. section 10.2 section 10.18

Figure 5.3 The classifier is the diagonal line that splits the happy and the sad points. The equation of this line is xaack = xbeep (or equivalently, xaack – xbeep = 0), because the line corresponds to all the points where the horizontal and the vertical coordinates are equal. The happy zone is the zone in which the number of appearances of aack is greater than or equal to the number of appearances of beep, and the sad zone is the zone in which the number of appearances of aack is less than that of beep.

Use the plot to connect word-count features with a decision boundary: points on opposite sides receive different sentiment labels. section 5.2

How Grokking Machine Learning, Second Edition covers it

How deep it goes
Explains definitions, feature-based examples, linear boundaries, weights, bias, iterative training, and neural-network extensions.
Examples
Uses spam email, cat-versus-dog labels, word-count sentiment, MNIST, CIFAR-10, and a circular two-class dataset.
What it assumes
Begins with labeled data, features, and categories without assuming prior machine learning knowledge.
What you can do after
Evaluate/understand whether a labeled prediction task calls for classification rather than regression.
How it connects
Connects supervised learning with labeled data, regression, perceptrons, neural networks, and generative machine learning.

Also explained in

Compare how 36 books explain classifier →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Grokking Machine Learning, Second Edition. The section links point to the parts of the book it draws on.