Suppose you know how often people sneeze when they have a cold, and how often they sneeze when they do not. Those two conditional distributions describe a model of sneezing; they do not, by themselves, tell you how common colds are. A likelihood function packages the model as a function from a condition to a distribution of possible observations. In C#, its shape is `Func<A, IDiscreteDistribution<B>>`. section 13.2 section 13.6
Listing 13.1 A likelihood function relating sneezing to having a cold
static IDiscreteDistribution<Sneezed> SneezedGivenCold(Cold c)
{
List<Sneezed> sneezed = [Sneezed.No, Sneezed.Yes];
return c == Cold.No ?
sneezed.ToCategorical(97, 3) :
sneezed.ToCategorical(14, 86); #1
}
The cold example returns a different categorical distribution for each condition: 86% sneeze with a cold, versus 3% without one. section 13.2
Pair the likelihood with a prior—say, 10% of people have colds—to obtain a joint distribution of conditions and observations. For each pair, multiply the condition’s prior probability by the observation’s conditional probability. The implementation scales weights to integers using the least common multiple of the conditional distributions’ total weights. With prior weights 90:10, the four joint weights for (no cold, no sneeze), (no cold, sneeze), (cold, no sneeze), and (cold, sneeze) are 873, 27, 14, and 86. section 13.2
Listing 13.3 Computing the joint distribution from a prior and a likelihood function
public static class Extensions
{
[...]
public static IDiscreteDistribution<(A, B)> ToJoint<A, B>(
this IDiscreteDistribution<A> prior,
Func<A, IDiscreteDistribution<B>> likelihood)
{
int lcm = prior.Support()
.Select(a => likelihood(a).TotalWeight())
.LCM(); #1
List<(A, B)> jointSupport = [];
List<int> jointWeight = [];
foreach (A a in prior.Support())
{
var db = likelihood(a);
foreach (B b in db.Support())
{
jointSupport.Add((a, b));
jointWeight.Add(
prior.Weight(a) * db.Weight(b) * lcm /
db.TotalWeight());
}
}
return jointSupport.ToCategorical(jointWeight);
}
}
`ToJoint` builds the tuple support and its weights from a prior and a likelihood function. section 13.2
Now reverse the question: given a sneeze, how likely is a cold? Filter the joint distribution to sneezing outcomes and project back to the cold condition. The remaining weights are 27 without a cold and 86 with one, giving a posterior cold probability of 86/113. This is Bayesian updating: the likelihood describes how observations depend on conditions, while the prior supplies how common those conditions were before the observation. section 13.3 section 13.6
Why a positive test is not 99.5% certainty
For a condition affecting 1 person in 30,000, a test with 99.5% sensitivity and specificity produces posterior weights of 199 for having the condition and 29,999 for not having it after a positive result. Despite the test’s accuracy, the resulting probability is only about 0.7%: the rare-condition prior matters. section 13.4
A likelihood can also ignore its input. Passing the same d20 distribution for either value of a first roll models two independent rolls. Their joint distribution contains 400 pairs; projecting each pair to its maximum gives the distribution for rolling with advantage. The same joint-distribution machinery therefore handles dependent and independent observations. section 13.5
