Chapter 4. Classifying with probability theory: naïve Bayes

In the first two chapters we asked our classifier to make hard decisions. We asked for a definite answer for the question “Which class does this data instance belong to?” Sometimes the classifier got the answer wrong. We could instead ask the classifier to give us a best guess about the class and assign a probability estimate to that best guess.

Probability theory forms the basis for many machine-learning algorithms, so it’s important that you get a good grasp on this topic. We touched on probability a bit in chapter 3 when we were calculating the probability of a feature taking a given value. We calculated the probability by counting the number of times the feature equals that value divided by the total number of instances in the dataset. We’re going to expand a little from there in this chapter.

4.1. Classifying with Bayesian decision theory

4.2. Conditional probability

4.3. Classifying with conditional probabilities

4.4. Document classification with naïve Bayes

4.5. Classifying text with Python

4.6. Example: classifying spam email with naïve Bayes

4.7. Example: using naïve Bayes to reveal local attitudes from personal ads

4.8. Summary