concept chain rule in category probabilistic programming
This is an excerpt from Manning's book Practical Probabilistic Programming.
I won’t describe how the calculations are performed using the rules of inference here. The code uses three rules of inference: the chain rule, the total probability rule, and Bayes’ rule. All these rules are explained in detail in chapter 9. For now, let’s point out two major problems with this code:
Figure 9.1. Inputs and output of each of the three rules of probabilistic inference. The chain rule lets you turn a set of conditional probability distributions into a joint probability distribution. The total probability rule lets you take a joint probability distribution over a set of variables and produce a distribution over a single variable. Bayes’ rule lets you “invert” a conditional probability distribution over an effect, given a cause, into a conditional probability distribution over the cause, given the effect.
The factor data structure and the way it represents a probabilistic model and a query. You’ll see that this is tied closely to the chain rule and the total probability rule you learned about in the preceding chapter.
The first key to understanding factors and factored algorithms is the chain rule, which you saw in chapter 9. Recall that the chain rule lets you compute the joint probability of an assignment to many variables as a product of conditional probabilities. The chain rule is closely associated with Bayesian networks and is essential to the definition of the probability distribution defined by the Bayesian network.