liveBook · Manning

Naive bayes classifier

As explained in Timeless Algorithms, The Foundational Ideas

A naive Bayes classifier is a probabilistic classification model that applies Bayes' theorem while assuming features are conditionally independent given the class.

A Naive Bayes classifier assigns an observation to the class with the most probable explanation of its features. It combines each class’s prior probability with the likelihood of observing those features to estimate a posterior probability. For an email containing words such as “free,” “offer,” and “urgent,” it weighs whether spam or a legitimate sender is the more probable source. section 2.4.1

The model’s simplifying assumption is that features are conditionally independent given the class. This lets it multiply individual feature likelihoods to form a joint likelihood. Though real features may be correlated, the assumption can make classification fast and effective. Despite its name, the model’s parameters are typically estimated from training data using maximum likelihood; Bayes’ theorem then yields class posteriors. section 2.4.1 section 3.6.2

How Timeless Algorithms, The Foundational Ideas covers it

How deep it goes
Explains the independence assumption and likelihood-based parameter estimation, but does not detail implementation.
Examples
Uses spam detection and emails with words such as “free,” “offer,” and “urgent”; also names sentiment analysis, document classification, and medical diagnosis.
What you can do after
Evaluate/understand how class posteriors support choosing the most probable class for an observation.
How it connects
Connects Bayes’ theorem, supervised learning, conditional independence, and maximum likelihood estimation.

Also explained in

Compare how 7 books explain naive bayes classifier →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Timeless Algorithms, The Foundational Ideas. The section links point to the parts of the book it draws on.