A categorical distribution chooses one value from a finite set of categories. Each category has a weight; its probability is that weight divided by the total weight. Unlike a fair die or an unfair coin, its values can be of any type and can have arbitrary relative weights. section 12.13
An unfair die with named faces
Start with a fair six-sided die and map its faces to colors. If one face maps to green, three to blue, and two to red, the resulting weights are green:1, blue:3, red:2. The projection must combine the weights of faces that map to the same color—not merely rename each sampled face. section 12.7
In C#, a projected distribution can compute those weights by grouping the underlying support by its projected value and summing each group's weights. Dividing all weights by their greatest common divisor keeps the same relative probabilities with smaller integers. Alternatively, `ToCategorical` pairs a list of category values with weights directly. section 12.7 section 12.11
The sampling algorithm matters when draws are frequent or weights are large:
- A quick-and-dirty approach becomes inefficient as total weight grows; an optimized ladder takes O(log n) time per sample. section 12.13
- Rejection sampling can have an arbitrarily bad worst case. The alias algorithm instead prepares rows containing at most two values, then chooses a row uniformly and samples within it. It uses O(n) extra storage, takes O(1) time per sample, and never rejects a draw. section 12.11 section 12.13
Zero-weight categories need not appear in samples; the alias algorithm handles them. An integer-weight implementation can reject negative weights and an all-zero distribution. Integer weights also create a later complication: constructing exact joint-distribution weights from a prior and conditional likelihoods requires accounting for their different total weights. section 12.11 section 13.2
From a prior to a joint distribution
Suppose the probability of a cold is 0.1, and the probability of sneezing given a cold is 0.86. The joint probability of both is 0.1 × 0.86 = 0.086. A joint distribution is itself categorical: its categories are tuples, such as `(Cold.Yes, Sneezed.Yes)`. With integer weights, `ToJoint` uses a common multiple of likelihood totals to construct the tuple weights. section 13.2
