liveBook · Manning

Independence

As explained in Grokking Machine Learning, Second Edition

Event independence is the property that the occurrence of one event does not change the probability of another event.

Two events are independent when knowing that one occurred does not change the probability of the other. For independent events E and F, the probability that both occur equals the product of their probabilities: P(E ∩ F) = P(E)P(F). section 8.7

Estimating women with brown eyes

Suppose an island has 1,000 inhabitants, half are women, and one-tenth have brown eyes. Without a joint count, assume gender and eye color are independent: (1/2)(1/10) = 1/20, giving an estimate of 50 women with brown eyes. The estimate may be wrong, but independence provides a reasonable calculation from limited information. section 8.7

Independence is useful in naive Bayes. If words such as lottery and sale are treated as independent, the probability that an email contains both can be estimated by multiplying their individual probabilities. This is a simplifying assumption rather than a claim that the words truly cannot influence one another. section 8.7

Test independence by asking whether the surrounding information changes the probability:

  • Separate coin throws and results from different dice can be independent, depending on the events being compared. section 8.16
  • Rain and the day of the week may be treated differently from rain and the month, because seasonal context can change the probability of rain. section 8.16
  • Naive Bayes applies the independence assumption to features such as symptoms or email words when estimating a diagnosis or spam probability. section 8.16 section 8.7
Figure 8.11 Say 20% of the emails contain the word lottery, and 10% of the emails contain the word sale. We make the naive assumption that these two words are independent of each other. Under this assumption, the percentage of emails that contain both words can be estimated as 2%, namely, the product of 20% and 10%.

Visualizes the product rule by estimating the percentage of emails containing both lottery and sale as the product of their separate percentages. section 8.7

How Grokking Machine Learning, Second Edition covers it

How deep it goes
Defines independence, gives the product rule, and works numerical and classification examples before practicing judgments.
Examples
Uses coins, dice, weather, gender and eye color, email words, and patient symptoms as examples and exercises.
What it assumes
Assumes basic events and probabilities while practicing conditional reasoning through concrete coin, dice, weather, and dataset questions.
What you can do after
Apply it via naive Bayes to estimate joint feature probabilities and reason about whether events are independent.
How it connects
Connects independence with probability, the product rule, Bayes’ theorem, and naive Bayes.

Also explained in

Compare how 9 books explain independence →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Grokking Machine Learning, Second Edition. The section links point to the parts of the book it draws on.