Suppose a test for a rare condition is 99.5% accurate and returns positive. Is there a 99.5% chance the person has the condition? Not necessarily. Before interpreting the result, we need the prior: how plausible the condition was before this test. For a condition affecting 1 in 30,000 people, that starting point matters enormously. section 13.4
A prior probability expresses belief before additional evidence arrives. If 1,000 of 10,000 surveyed people have a cold, the prior probability that a randomly chosen person has one is P(C) = 0.1. In a program, a prior can be a distribution over possible conditions rather than a single number: here, 90% no cold and 10% cold. section 13.1 section 13.2
Table 13.1 A survey of 10,000 respondents organized in a marginal table
| Cold: YES |
Cold: NO |
Total |
|
|---|---|---|---|
| Sneezed: YES |
860 |
270 |
1,130 |
| Sneezed: NO |
140 |
8,730 |
8,870 |
| Total |
1,000 |
9,000 |
10,000 |
The survey’s Total row supplies the cold prior; its sneeze counts also let us compare that prior with belief after observing a sneeze. section 13.1
The prior is only the starting distribution. A likelihood function describes how probable an observation is under each condition: P(S|C) = 0.86 for a sneeze given a cold, versus P(S|not C) = 0.03 without one. Multiplying each prior probability by its corresponding likelihood produces joint probabilities. Thus P(C and S) = 0.1 × 0.86 = 0.086. section 13.1 section 13.2
Once a sneeze is observed, discard joint outcomes without a sneeze and compare those remaining. The survey has 860 people who both sneezed and had a cold, but also 270 who sneezed without one. The updated probability of a cold is 860/1,130, about 76%. This posterior answers a different question from the 10% prior: what should we believe after seeing this particular evidence? In general, P(A|B) = P(A)P(B|A)/P(B). section 13.1 section 13.3
For finite categorical distributions, the same reasoning maps directly to C# operations:
- Give each possible cause a prior weight with ToCategorical; for the cold example, the no-cold and cold weights are 90 and 10. section 13.2
- Pass the prior and a likelihood function to ToJoint. Each cause–observation pair receives a weight proportional to its joint probability; the implementation uses a least common multiple to accommodate integer likelihood weights. section 13.2
- Use Posterior to filter the joint distribution to an observed effect and project back to causes. For a sneeze, the resulting no-cold and cold weights are 27 and 86. section 13.3
Why an accurate positive can be weak evidence
Give the rare condition prior weights of 29,999 without it and 1 with it. A test with 99.5% sensitivity and specificity still produces posterior weights of 29,999 without the condition and 199 with it after a positive result. The probability of having the condition is only about 0.7%: false positives among the much larger unaffected group outweigh true positives. Test accuracy alone cannot supply the posterior. section 13.4 section 13.6
A prior need not stay fixed. After observing evidence, the posterior distribution can serve as the starting distribution for another update. With prior coin weights of 999 normal to 1 double-headed, ten successive heads yield weights of 999 to 1,024. Repeated evidence has nearly overcome a strong initial preference for the normal coin. section 13.4
