liveBook · Manning

Bernoulli naive Bayes

As explained in Machine Learning Algorithms in Depth

A probabilistic classification model that uses Bernoulli likelihoods for binary features and assumes conditional independence given the class.

Bernoulli naive Bayes is a probabilistic classifier for binary features. In document classification, each xᵢⱼ records whether word j is absent or present in document i. Conditional independence given the class makes the class-conditional density a product of one-dimensional Bernoulli densities. section 5.5

For C classes and a dictionary of size D, θⱼc parameterizes word presence for class c, while π gives class-label priors. The model has O(DC) learnable parameters. Estimation maximizes log likelihood subject to ∑πc = 1; a Lagrangian handles this constraint, and differentiating with respect to πc yields its fitted form. section 5.5

A from-scratch treatment follows this workflow:

  • Represent documents with binary word-presence variables and organize parameters by dictionary term and class. section 5.5
  • Derive class-conditional probabilities from the conditional-independence assumption, then optimize their log likelihood. section 5.5

Document classification

A document is converted into word-presence features and classified by comparing class probabilities. This supports applications such as email spam detection and document classification; the implementation example uses the 20 Newsgroups dataset. section 5.0

How Machine Learning Algorithms in Depth covers it

How deep it goes
Derives the likelihood and constrained parameter estimates, then implements the classifier from scratch.
Examples
Uses binary word indicators, document classification, email spam detection, and the 20 Newsgroups dataset.
What it assumes
Assumes computer-science fundamentals while developing supervised-learning and classification algorithms.
What you can do after
Build it from scratch and apply it to document classification with Python-based preprocessing and evaluation.
How it connects
Connects supervised learning, classification, Bernoulli distributions, Dirichlet priors, and Gaussian or multinomial Naive Bayes variants.
Cover of Machine Learning Algorithms in Depth
liveBookmanning.com
Read Machine Learning Algorithms in Depth on liveBook Get the book at manning.com

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Machine Learning Algorithms in Depth. The section links point to the parts of the book it draws on.