A posterior probability is a belief about an unknown cause after observing evidence. If A is the cause and B is the observation, Bayes’ theorem gives P(A|B) = P(A)P(B|A)/P(B). The prior P(A) expresses belief before the observation; the likelihood P(B|A) describes how likely the observation is under that cause. The posterior P(A|B) updates the belief without confusing an effect’s likelihood with the probability of its cause. section 13.1 section 13.3
A sneeze changes the odds
Suppose 10% of people have a cold. Among people with colds, 86% recently sneezed; among people without colds, 3% did. In a survey of 10,000 people, 860 of the 1,130 who sneezed had a cold. Observing a sneeze therefore changes the estimated probability of a cold from 10% to 860/1,130, or 76.1%. The denominator matters: the question is about people who sneezed, not just people with colds who sneezed. section 13.1 section 13.3
Table 13.1 A survey of 10,000 respondents organized in a marginal table
| Cold: YES |
Cold: NO |
Total |
|
|---|---|---|---|
| Sneezed: YES |
860 |
270 |
1,130 |
| Sneezed: NO |
140 |
8,730 |
8,870 |
| Total |
1,000 |
9,000 |
10,000 |
The marginal table makes the update visible: restrict attention to the sneezed-yes row, then compare its cold-yes count with its row total. section 13.1
For a discrete distribution, the same calculation can be built from reusable operations. Combine a prior distribution over A with a likelihood function from A to distributions over B to form a joint distribution of (A, B). Filter that joint distribution to the observed B, then project out A. The resulting function takes an observation and returns a posterior distribution over possible causes. section 13.3
Why an accurate test can yield a small posterior
A rare syndrome affects 1 person in 30,000. A test is positive for 995 of 1,000 affected people and negative for 995 of 1,000 unaffected people. After a positive result, the posterior weights are 199 for having the syndrome and 29,999 for not having it—about a 0.7% probability of having it. Test accuracy alone cannot supply the posterior; the rare prior remains consequential. Repeated evidence can also make a large update: ten heads shift a 1-in-1,000 prior for a double-headed coin to posterior weights of 1,024 versus 999 for a normal coin. section 13.4
Continuous unknowns use the same prior-times-likelihood idea, but obtaining a distribution that can be sampled requires another step. For a coin whose unknown fairness lies between 0 and 1, a clamped normal distribution supplies the prior and a binomial distribution models observed heads. The unnormalized posterior weight at each fairness value is prior.Weight(value) × likelihood(value).Weight(observation). Metropolis uses those weights, the prior as an initial distribution, and a normal proposal centered on the current sample to produce posterior samples. section 16.3 section 16.5
Listing 16.8 Completing the implementation of continuous posterior sampling
static class Extensions
{
public static Func<B, IContinuousDistribution> Posterior<B>(
this IContinuousDistribution prior,
Func<double, IWeightedDistribution<B>> likelihood,
double stddev = 1.0)
{
return (B evidence) =>
{
Func<double, double> posteriorPDF = a =>
prior.Weight(a) * likelihood(a).Weight(evidence);
Func<double, IContinuousDistribution> proposal =
current => Normal.Distribution(current, stddev);
return Metropolis.Distribution(posteriorPDF, prior, proposal);
};
}
}
The C# implementation connects the continuous posterior weight function to Metropolis, including its initial and proposal distributions. section 16.5
