Suppose someone is 165 cm tall and has a 44 cm femur. To ask how tall that same person would be if their femur were 46 cm, a causal model must first infer the unobserved factors compatible with the measurements. Variational inference can perform this abduction step; it is a probabilistic inference technique, not a causal assumption. section 9.3 section 9.3.3
Variational inference replaces a target distribution that is difficult to infer directly with an approximating distribution. A stochastic guide function generates samples from that approximation. Its trainable parameters are optimized to make the approximation close to the target. Unlike an automatic graphical-model inference routine, a hand-written guide lets the developer choose how to represent the proposed values. section 2.2.5 section 9.3.3
Abduction before a counterfactual intervention
In a structural causal model of sex, femur length, and height, condition on femur length 44 cm and height 165 cm. Train a guide Q over the model’s exogenous variables to approximate their conditional distribution given those observations. Then intervene to set femur length to 46 cm and generate heights using samples from the inferred exogenous distribution. Sampling from the original exogenous distribution instead answers the different question of height under that intervention without conditioning on this person’s measurements. section 9.3 section 9.3.1 section 9.3.3
A Pyro implementation pairs a conditioned model with a guide in stochastic variational inference (`SVI`), uses `Trace_ELBO` as the loss and `Adam` for optimization, and calls `svi.step` repeatedly. The fitted guide parameters can then define distributions for the exogenous variables. When deterministic assignment functions must participate in gradient-based inference, one implementation samples endogenous variables from narrow normal distributions rather than exact point-mass distributions. section 9.3.1 section 9.3.3
The same approach also trains latent-variable image models: an encoder supplies a distribution over a latent value, and a decoder uses a sampled value to reconstruct an image. Reconstruction error alone can overfit, so training uses a probabilistic objective. Variational inference works with neural networks and latent variables, and can also support causal workflows that estimate an average treatment effect after fitting a generative model to data. section 5.2.3 section 11.6
Listing 9.15 Implementing the abduction step with variational inference
from pyro.infer import SVI, Trace_ELBO
from pyro.optim import Adam
pyro.util.set_rng_seed(123) #1
pyro.clear_param_store() #2
svi = SVI( #3
model=conditioned_model,
guide=guide,
optim=Adam({"lr": 0.003}), #4
loss=Trace_ELBO() #5
)
losses = [] #6
num_steps = 5000 #7
for t in range(num_steps): #7
losses.append(svi.step(exogenous)) #7
plt.plot(losses) #8
plt.title("Loss During Training") #8
plt.xlabel("step") #8
plt.ylabel("loss") #8
The Pyro listing shows the conditioned model and guide passed to `SVI`, followed by repeated optimization steps and a loss plot. section 9.3.3
