The chain rule factorizes a joint probability into conditional probabilities. For three variables, P(X, Y, Z) can be written as P(X)P(Y|X)P(Z|X,Y), or with any other variable ordering. The ordering changes the factors, not the validity of the factorization. section 2.1.4
This turns one difficult modeling object into separate tasks: represent each conditional distribution and combine them computationally. Conditional independence can simplify factors further; for example, if Y is conditionally independent of Z given X, use P(Z)P(X|Z)P(Y|X). section 2.1.4 section 2.2.3
The factorization supports several practical workflows:
- Represent conditional factors with canonical distributions, such as a normal distribution whose parameters depend on other variables through a linear function or neural network. section 2.1.7
- Generate a joint sample sequentially: sample Z, sample X conditioned on Z, then sample Y conditioned on X. section 2.2.3
- Implement the resulting process with BayesianNetwork in pgmpy or composed distributions and sampling statements in Pyro. section 2.2.3
Sequential sampling process
A Pyro process samples z from a Gamma distribution, x from a Poisson distribution conditioned on z, and y from a Bernoulli distribution whose parameter depends on x. The resulting values form a sample from the modeled joint process. section 2.2.3
Listing 2.5 Creating a random process in pgmpy and Pyro
from pgmpy.factors.discrete.CPD import TabularCPD
from pgmpy.models import BayesianNetwork
from pgmpy.sampling import BayesianModelSampling
PZ = TabularCPD( #1
variable='Z', #1
variable_card=2, #1
values=[[.65], [.35]], #1
state_names = { #1
'Z': ['0', '1'] #1
}) #1
PXgivenZ = TabularCPD( #2
variable='X', #2
variable_card=2, #2
values=[ #2
[.8, .6], #2
[.2, .4], #2
], #2
evidence=['Z'], #2
evidence_card=[2], #2
state_names = { #2
'X': ['0', '1'], #2
'Z': ['0', '1'] #2
}) #2
PYgivenX = TabularCPD( #3
variable='Y', #3
variable_card=3, #3
values=[ #3
[.1, .8], #3
[.2, .1], #3
[.7, .1], #3
], #3
evidence=['X'], #3
evidence_card=[2], #3
state_names = { #3
'Y': ['1', '2', '3'], #3
'X': ['0', '1'] #3
}) #3
model = BayesianNetwork([('Z', 'X'), ('X', 'Y')]) #4
model.add_cpds(PZ, PXgivenZ, PYgivenX) #5
generator = BayesianModelSampling(model) #6
generator.forward_sample(size=1) #7
Inspect the pgmpy implementation that encodes P(Z), P(X|Z), and P(Y|X) as conditional probability tables. section 2.2.3
