A joint probability distribution assigns probabilities to combinations of outcomes. For a person’s cold status C and recent sneezing S, each outcome is a pair: (cold, sneezed), (cold, did not sneeze), and so on. The probability of a pair comes from a prior and a conditional probability: P(C & S) = P(C) × P(S|C). If 10% of people have colds and 86% of those people sneeze, the joint probability of having a cold and sneezing is 8.6%. That is not the same as P(C|S), the probability of a cold given a sneeze. section 13.1
Table 13.1 A survey of 10,000 respondents organized in a marginal table
| Cold: YES |
Cold: NO |
Total |
|
|---|---|---|---|
| Sneezed: YES |
860 |
270 |
1,130 |
| Sneezed: NO |
140 |
8,730 |
8,870 |
| Total |
1,000 |
9,000 |
10,000 |
The survey table makes the distinction visible: 860 of all 10,000 respondents both had a cold and sneezed, while 860 of the 1,130 who sneezed had a cold. section 13.1
In C#, a joint distribution can be a categorical distribution whose categories are tuples. Start with a prior distribution for C and a likelihood function that returns a distribution for S given each value of C. Sampling the prior and then its corresponding likelihood generates paired outcomes. To represent their probabilities exactly instead, `ToJoint` visits every supported pair and assigns it a weight proportional to the prior weight multiplied by the conditional weight. Because the inputs use integer weights, the implementation uses the least common multiple of the likelihood distributions’ total weights to keep the resulting weights integral. section 13.2
Listing 13.3 Computing the joint distribution from a prior and a likelihood function
public static class Extensions
{
[...]
public static IDiscreteDistribution<(A, B)> ToJoint<A, B>(
this IDiscreteDistribution<A> prior,
Func<A, IDiscreteDistribution<B>> likelihood)
{
int lcm = prior.Support()
.Select(a => likelihood(a).TotalWeight())
.LCM(); #1
List<(A, B)> jointSupport = [];
List<int> jointWeight = [];
foreach (A a in prior.Support())
{
var db = likelihood(a);
foreach (B b in db.Support())
{
jointSupport.Add((a, b));
jointWeight.Add(
prior.Weight(a) * db.Weight(b) * lcm /
db.TotalWeight());
}
}
return jointSupport.ToCategorical(jointWeight);
}
}
The `ToJoint` implementation shows how a prior and likelihood function become one distribution over pairs, including the integer-weight scaling. section 13.2
Once the pairs are available, the same distribution supports several operations:
- Condition on an observation by filtering pairs to those with the observed second value, then project the first value. This yields a posterior distribution; for someone who sneezed, the cold-to-no-cold weights are 86 to 27. section 13.3
- Combine independent quantities by using a likelihood that ignores its input. Two independent d20 rolls produce 400 possible pairs; projecting each pair to its maximum gives the distribution for rolling with advantage. section 13.5
Why an accurate test can mislead
Suppose a condition has prior odds of 1 in 30,000 and a test has 99.5% sensitivity and specificity. The joint distribution assigns weights of 199 to (condition, positive) but 29,999 to (no condition, positive). Filtering to positive results reveals why most positive tests are false positives despite the test’s accuracy: the rare condition’s prior still matters. section 13.4
