liveBook · Manning

Conditional probability

As explained in Math and Architectures of Deep Learning

The probability of an event computed under the assumption that another event or condition is already known to hold.

Conditional probability measures how likely an event is when another event or condition is already known to hold. Writing p(X = x | Y = y) means “the probability of X = x given Y = y.” Knowing Y restricts the population to cases satisfying that condition, so the reference population changes. section 6.1.2

For events with nonzero conditioning probability, p(X, Y) = p(X | Y)p(Y), so p(X | Y) = p(X, Y) / p(Y). The numerator is the joint probability, while the denominator is the marginal probability of the condition. Bayes’ theorem reverses the conditioning direction by relating p(X | Y) to p(Y | X), joint probability, and marginal probability. section 6.1.3 section 6.10

Height and weight in Statsville

Suppose a subject’s height is between 160 and 183 cm, represented by H = F2. The question “What is the probability that weight exceeds 90 kg?” becomes p(W = E3 | H = F2). The relevant population is the 48,000 residents in the F2 row, not the entire population of 100,000 residents. section 6.1.2

Table 6.2 Example population sizes and joint, marginal, and conditional probabilities for variables W = {E1, E2, E3} and H = {F1, F2, F3} weights and heights of adult Statsville residents). (This is table 6.1 with conditional probabilities added.)

Less than 60 kg (E1)

Between 60 and 90 kg (E2)

More than 90 kg (E3)

Marginals for Fs

Less than 160 cm (F1)

pop. = 20,000

p(E1, F1) = 0.2

p(E1| F1) = p(E1, F1) / p(F1) = 0.77

p(F1| E1) = p(E1, F1) / p(E1) = 0.77

pop. = 4,000

p(E2, F1) = 0.04

p(E2| F1) = p(E2, F1) / p(F1) = 0.154

p(F1| E2) = p(E2, F1) / p(E2) = 0.083

pop. = 2,000

p(E3, F1) = 0.02

p(E3| F1) = p(E3, F1) / p(F1) = 0.077

p(F1| E3) = p(E3, F1) / p(E3) = 0.077

pop. = 26,000;

p(F1) = 0.2

+ 0.04 + 0.02 = 0.26

Between 160 cm and 183 cm (F2)

pop. = 4,000

p(E1, F2) = 0.04

p(E1| F2) = p(E1, F2) / p(F2) = 0.083

p(F2)| E1) = p(E1, F2) / p(E1) = 0.154

pop. = 40,000

p(E2, F2) = 0.4

p(E2| F2) = p(E2, F2) / p(F2) = 0.83

p(F2)| E2) = p(E2, F2) / p(E2) = 0.83

pop. = 4,000

p(E3, F2) = 0.04

p(E3| F2) = p(E3, F2) / p(F2) = 0.083

p(F2)| E3) = p(E3, F2) / p(E3) = 0.154

pop. = 48,000;

p(F2) = 0.04 + 0.4 + 0.04 = 0.48

More than 183 cm (F3)

pop. = 2,000

p(E1, F3) = 0.02

p(E1|F3) = p(E1, F3) / p(F3) = 0.077

p(F3|E1) = p(E1, F33) / p(E1) = 0.077

pop. = 4,000

p(E2, F3) = 0.04

p(E2|F3) = p(E2, F3) / p(F3) = 0.154

p(F3|E2) = p(E2, F33) / p(E2) = 0.083

pop. = 20,000

p(E3, F3) = 0.2

p(E3|F3) = p(E3, F3) / p(F3) = 0.77

p(F3|E3) = p(E3, F33) / p(E3) = 0.77

pop. = 26,000;

p(F3) = 0.02 + 0.04 + 0.2 = 0.26

Marginals for Es

p(E1) = 0.2 + 0.04 + 0.02 = 0.26

p(E2) = 0.04 + 0.4 + 0.04 = 0.48

p(E3) = 0.02 + 0.04 + 0.2 = 0.26

Total pop.= 100,000;

Total prob = 1

Use the joint-probability table to compare conditional probabilities with marginal probabilities and probabilities conditioned in the reverse direction. section 6.1.2

In machine learning, conditional probability p(x | θ) is the likelihood of an input given model parameters θ, while p(θ | x) is the posterior probability of parameters given the input. Independence is the special case where conditioning does not change a probability: p(X | Y) = p(X). section 6.1.3 section 6.10

How Math and Architectures of Deep Learning covers it

How deep it goes
Defines conditioning, derives the joint-to-conditional ratio, and connects it to Bayes’ theorem and independence.
Examples
Uses a height-and-weight population table to calculate and compare joint, marginal, and conditional probabilities.
What you can do after
Evaluate conditional probabilities and interpret likelihoods, posteriors, and independence in Bayesian models.
How it connects
Connects conditional probability to joint probability, marginal probability, Bayes’ theorem, likelihood, posterior probability, and independence.
Cover of Math and Architectures of Deep Learning
liveBookmanning.com
Read Math and Architectures of Deep Learning on liveBook Get the book at manning.com

Also explained in

Compare how 11 books explain conditional probability →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Math and Architectures of Deep Learning. The section links point to the parts of the book it draws on.