Is this email spam or not? A classifier learns from examples with known labels, then predicts a category for a new example. Here, each email is an example, its words or other properties are features, and “spam” or “ham” is its label. This is supervised learning because the training examples come with labels. section 2.2 section 5.5
A classifier predicts a category, not an unrestricted number. A house-price model predicts a number and is therefore a regression model. Many classifiers answer a yes-or-no question, but a classifier can also choose among several categories, such as animal types or apps to recommend. section 2.2 section 5.0
Different classifiers turn features into predictions in different ways:
- A perceptron gives each feature a weight, adds a bias, and predicts yes when the resulting score is at least zero. For sentence sentiment, words can receive positive or negative scores. Visually, the model tries to separate two classes of points with a line. section 5.6
- Naive Bayes estimates a label’s probability from observed features. In a spam filter, words provide evidence that changes the estimated probability that an email is spam. section 8.0 section 8.2
- A decision tree asks questions about features and predicts from the answers. At a leaf, it can choose the most common label among the training examples that reached it. section 9.2 section 9.8
Which question makes a better classifier?
Suppose you can ask app users only one question before predicting which app they downloaded. Splitting six training examples by platform gives four correct predictions; splitting by age gives five. Accuracy—the fraction of examples classified correctly—favors age on this dataset. A decision tree can continue splitting a group when another question helps separate its labels. section 9.2
Figure 9.9 Classifier 1 uses platform, and classifier 2 uses age. To make the prediction at each leaf, each classifier picks the most common label among the samples in that leaf. Classifier 1 is correct four out of six times, and classifier 2 is correct five out of six times. Therefore, based on accuracy, classifier 2 is better.
Compare the platform and age splits: each leaf predicts its most common label, making the difference in accuracy visible. section 9.2
To try this yourself, take a small labeled set of emails. Pick one feature, such as whether an email contains a particular word, and make a simple spam-or-ham rule. Count how often it predicts the recorded labels correctly. Then try another feature and compare the counts; deciding which rule works best is part of building a useful classifier. section 1.3 section 9.2
