liveBook · Manning

Multiplication rule

As explained in Grokking Machine Learning, Second Edition

The product rule for independent probabilities states that the probability of independent events occurring together is the product of their individual probabilities.

The multiplication rule finds the chance that two events happen together. Start with one branch of a situation, then multiply by the chance of the next event along that branch. For example, if 1/5 of emails are spam and 3/4 of spam emails contain “lottery,” then the chance of an email being both spam and containing “lottery” is (1/5)(3/4) = 3/20, or 0.15. section 8.6

When the second probability depends on the first event, use a conditional probability: P(E ∩ F) = P(E | F) × P(F). In plain language, multiply the chance of F by the chance of E after F has occurred. The result is a joint probability—the probability of both events together. section 8.6

For independent events, neither event changes the chance of the other. The conditional part is then unnecessary: multiply their individual probabilities. If half a population is women and one tenth has brown eyes, independence estimates women with brown eyes as (1/2)(1/10) = 1/20; among 1,000 people, that is 50 people. section 8.7

Two words in an email

Suppose 20% of emails contain “lottery” and 10% contain “sale.” If the words are assumed independent, the estimated chance that an email contains both is 20% × 10% = 2%. This independence assumption is useful when the joint count is unavailable, but it is an estimate rather than a guaranteed fact. section 8.7

Figure 8.8 The same tree from figure 8.6, but now with probabilities. From the root, two branches emerge, one for spam emails and one for ham emails. In each one, we record the corresponding probability. Each branch again splits into two leaves: one for emails containing the word lottery, and one for emails not containing it. In each branch we record the corresponding probability. Notice that the product of these probabilities is the probability at the right of each leaf. For example, for the top leaf, 1/5 3/4 = 3/20 = 0.15.

Follow each path from spam or ham to whether “lottery” appears. Multiplying the probabilities on a path gives the probability at its leaf. section 8.6

The same idea also supports the Gini impurity index. If two draws are independent, the chance that both selected balls have color i is pᵢ². Adding these same-color probabilities and subtracting from 1 gives the chance of different colors. section 9.3

How Grokking Machine Learning, Second Edition covers it

How deep it goes
Defines conditional and independent forms, then works through numerical email, population, and color-draw calculations.
Examples
Uses spam emails with “lottery” and “sale,” population eye-color estimates, and independently drawn colored balls.
What it assumes
Introduces conditional probability and independence through counted examples rather than assuming machine-learning background.
What you can do after
Apply it to calculate or estimate the probability that two events occur together.
How it connects
Connects conditional probability, independence, Bayes’ theorem, naive Bayes, and Gini impurity.

Also explained in

Compare how 4 books explain multiplication rule →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Grokking Machine Learning, Second Edition. The section links point to the parts of the book it draws on.