liveBook · Manning

Joint probability distribution

As explained in Math and Architectures of Deep Learning

The probability that a particular combination of values for two or more random variables occurs together.

A joint probability is the probability that particular values of two or more random variables occur together. A joint probability distribution assigns one probability to every possible combination; all cells sum to 1. section 5.4 section 6.1.1

Use these properties to read and validate a joint distribution:

  • For independent events, multiply the individual probabilities: p(Ei, Gj) = p(Ei)p(Gj). section 5.4
  • Recover a marginal probability by summing across every value of the other variable; row and column totals provide the marginals. section 5.4.1
  • For dependent variables, the joint probability generally differs from the product of the marginals. section 5.4.2 section 6.1.1

Quantized resident variables

Suppose weight is divided into E1, E2, and E3, while home distance is divided into G1, G2, and G3. With independent variables, p(E2) = 0.5 and p(G2) = 0.6 give p(E2, G2) = 0.30. Summing the G2 row recovers p(G2) = 0.6. By contrast, a weight-height table concentrates probability along its diagonal, showing dependence. section 5.4 section 5.4.1 section 6.1.1

Table 6.1 Example population sizes and joint probability distribution for variables W = {E1, E2, E3} and H = {F1, F2, F3} weights and heights of adult Statsville residents), showing marginal probabilities

Less than 60 kg (E1)

Between 60 and 90 kg (E2)

More than 90 kg (E3)

Marginals for Fs

Less than 160 cm (F1)

pop. = 20,000

p(E1, F1) = 0.2

pop. = 4,000

p(E2, F1) = 0.04

pop. = 2,000

p(E3, F1) = 0.02

pop. = 26,000;

p(F1) = 0.2 + 0.04 + 0.02 = 0.26

Between 160 cm and 183 cm (F2)

pop. = 4,000

p(E1, F2) = 0.04

pop. = 40,000

p(E2, F2) = 0.4

pop. = 4,000

p(E3, F2) = 0.04

pop. = 48,000;

p(F2) = 0.04 + 0.4 + 0.04 = 0.48

More than 183 cm (F3)

pop. = 2,000

p(E1, F3) = 0.02

pop. = 4,000

p(E2, F3) = 0.04

pop. = 20,000

p(E3, F3) = 0.2

pop. = 26,000;

p(F3) = 0.02 + 0.04 + 0.2 = 0.26

Marginals for Es

p(E1)

= 0.2 + 0.04 + 0.02 = 0.26

p(E2)

= 0.04 + 0.4 + 0.04 = 0.48

p(E3)

= 0.02 + 0.04 + 0.2 = 0.26

Total pop. = 100,000;

Total prob = 1

Inspect the complete weight-height table to compare individual joint cells, marginal totals, and the evidence for dependence. section 6.1.1

How Math and Architectures of Deep Learning covers it

How deep it goes
Develops cells, normalization, independent products, marginal sums, and dependent-variable contrasts through worked discrete tables.
Examples
Uses quantized weight, home-distance, and height categories with numeric population tables.
What it assumes
Assumes basic events and random variables, while developing joint and marginal ideas through discrete examples.
What you can do after
Evaluate joint tables, recover marginals, and test independence by comparing joint probabilities with marginal products.
How it connects
Links joint probability to marginal probability, conditional probability, independence, and Bayes' theorem.
Cover of Math and Architectures of Deep Learning
liveBookmanning.com
Read Math and Architectures of Deep Learning on liveBook Get the book at manning.com

Also explained in

Compare how 5 books explain joint probability distribution →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Math and Architectures of Deep Learning. The section links point to the parts of the book it draws on.