liveBook · Manning

Naive bayes classifier

As explained in Grokking Machine Learning, Second Edition

A probabilistic classifier that applies Bayes' rule while assuming conditional independence among features given the class.

A naive Bayes classifier predicts a label as a probability. For spam detection, it estimates how likely an email is to be spam from its words, then sends high-probability messages to the spam folder. section 8.0 section 8.3

Start with a prior: the probability of a label when little is known. Each observed feature supplies new information, producing a posterior probability. Bayes’ theorem formalizes that update. section 8.2 section 8.15

The “naive” step makes a useful simplifying assumption: feature events are independent. The classifier can then multiply the probabilities associated with several words and combine them with the spam and ham priors. section 8.7 section 8.8

From familiar words to a spam score

Words such as lottery and sale can push an email toward spam, while mom can pull it back toward ham. In a worked three-word case, adding mom reduces the spam probability from 94.74% to 87.80%. section 8.8

predict_naive_bayes('Hi mom how are you')
0.12554358867163865
 predict_naive_bayes('meet me at the lobby of the hotel at nine am')
0.00006964603508395
 predict_naive_bayes('buy cheap lottery easy money now')
0.9999734722659664
 predict_naive_bayes('asdfgh')
0.2388268156424581

Try contrasting ordinary messages, promotional language, and an unknown word. An email with no dictionary words returns the prior probability. section 8.13

A hand-built implementation lowercases an email, keeps its distinct words, looks up word statistics for spam and ham, multiplies them, and returns the normalized spam probability. Scaling factors keep products from becoming too small for Python. section 8.13 section 8.9

How Grokking Machine Learning, Second Edition covers it

How deep it goes
Works through priors, posteriors, independence, multiword calculations, and a hand-coded predictor.
Examples
Uses spam and ham emails, lottery-sale-mom word probabilities, and a COVID-19 symptom diagnosis exercise.
What it assumes
Introduces priors, events, and posteriors before deriving the classifier.
What you can do after
Build it from scratch to score emails using learned word frequencies and class priors.
How it connects
Connects Bayes’ theorem, conditional probability, prior probability, posterior probability, independent events, and classification.

Also explained in

Compare how 7 books explain naive bayes classifier →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Grokking Machine Learning, Second Edition. The section links point to the parts of the book it draws on.