Suppose an email contains the word “lottery.” Is it spam? A naive Bayes classifier estimates the probability of a label, such as spam, from observed features, such as words. It starts with the probability of spam before reading the email—the prior—and updates that estimate using what the words tell us. The updated probability is the posterior. section 8.0 section 8.2
One word changes the estimate
Imagine 100 emails: 20 are spam, so the prior probability of spam is 20%. Of the 20 emails containing “lottery,” 15 are spam. Once you know an email contains that word, the estimated probability of spam becomes 15 ÷ 20 = 75%. The relevant comparison is now among emails containing “lottery,” not among all emails. section 8.2
What if the email contains both “lottery” and “sale”? Naive Bayes makes its simplifying assumption: within each class, treat the observed words as independent. Estimate the chance of seeing both words in spam by multiplying their separate chances in spam; do the same for ham. Multiply each result by that class’s prior, then divide the spam result by the sum of both results to obtain the spam probability. For words x₁ through xₙ, the assumption is P(x₁, …, xₙ | spam) = P(x₁ | spam) × … × P(xₙ | spam). It makes a many-word calculation manageable, but it remains an assumption about the words. section 8.2
Figure 8.11 Say 20% of the emails contain the word lottery, and 10% of the emails contain the word sale. We make the naive assumption that these two words are independent of each other. Under this assumption, the percentage of emails that contain both words can be estimated as 2%, namely, the product of 20% and 10%.
The diagram illustrates the simplifying independence assumption by estimating how often two words occur together from their separate frequencies. section 8.2
To build a word-based spam classifier by hand:
- Load emails labeled spam or ham. Lowercase each email and keep a set of its words, so repeated occurrences count only once. section 8.3
- Count spam and ham emails to find the priors. For each known word, count the spam and ham emails containing it; the Python example starts each word’s counts at one. section 8.3
- For a new email, combine its known words’ class-specific estimates with the priors, then turn the two results into a spam probability. The example prediction function skips words absent from its stored counts. section 8.3
Try a prediction
A hand-coded classifier gives “buy cheap lottery easy money now” a spam probability of about 0.99997, while an unfamiliar word by itself returns the prior, about 0.239. For another hands-on test, use symptoms such as cough and fever to estimate whether a patient is sick; an unmentioned symptom is unknown, not automatically absent. section 8.3 section 8.5
A probability is useful for sorting likely spam, but predictions still need evaluation. One workflow trains Scikit-Learn’s GaussianNB alongside other classifiers, chooses among models using validation data and metrics such as accuracy and F1-score, and reserves the test set until the end. section 8.2 section 13.4
