Suppose you know a person’s femur length and height, but need to estimate how tall they would have been with a different femur length. The counterfactual first requires *abduction*: inferring the unobserved factors consistent with the measurements. A Pyro guide function supplies a stochastic process for that inference. It generates candidate values from an approximating distribution rather than requiring direct samples from the target posterior. section 9.3.3
For observed data x and latent variables z, the guide represents Q(z), an approximation to the posterior P(z | x). Its sampling statements specify which values to propose; trainable parameters determine the proposal’s shape. In stochastic variational inference (SVI), optimization adjusts those parameters to bring Q closer to the posterior. This differs from a model function, which specifies the generative process and the distribution being inferred. section 9.3.3 section 2.2.5
The implementation pattern is:
- Define trainable distribution parameters with `pyro.param`, applying constraints where needed; sample proposed latent values with `pyro.sample`. section 9.3.3
- Condition the model on observations, then pass the conditioned model and guide to `SVI` with an optimizer and `Trace_ELBO`. Repeated `svi.step(...)` calls train the guide. section 9.3.3 section 11.6.5
- For a large latent-variable model, use an encoder inside the guide to turn observed inputs into parameters for a latent distribution. section 11.6.5
Abducing factors for a height counterfactual
With femur length observed at 44 cm and height at 165 cm, a guide proposes values for three exogenous variables. Bernoulli and Normal distributions have trainable parameters. After SVI fits the guide against the conditioned model, the resulting distributions approximate the exogenous posterior needed before asking about a 46 cm femur. section 9.3.3
Listing 9.13 Specifying the guide function for variational inference
import torch.distributions.constraints as constraints
from pyro.primitives import param
from pyro.distributions import Delta
def guide(exogenous): #1
p = param("p", tensor(.5), #2
constraint=constraints.unit_interval) #2
n_sex = sample("N_sex", Bernoulli(p)) #2
sex = sample("sex", Bernoulli(n_sex)) #3
n_femur_loc = param("n_femur_loc", tensor(0.0)) #4
n_femur_scale = param( #4
"n_femur_scale", #4
tensor(1.0), #4
constraint=constraints.positive #4
) #4
femur_dist = Normal(n_femur_loc, n_femur_scale) #5
n_femur = sample("N_femur", femur_dist) #5
n_height_loc = param("n_height_loc", tensor(0.0)) #5
n_height_scale = param( #5
"n_height_scale", #5
tensor(1.0), #5
constraint=constraints.positive #5
) #5
height_dist = Normal(n_height_loc, n_height_scale) #5
n_height = sample("N_height", height_dist) #5
femur = sample("femur", Delta(n_femur)) #6
height = sample("height", Delta(n_height)) #6
A concrete guide declares constrained trainable parameters and samples proposed exogenous values. section 9.3.3
A guide is not limited to SVI. In importance resampling, a proposal generates samples that are weighted by their probability under the conditioned model relative to the proposal. The choice of proposal therefore matters even when its parameters are not trained by an ELBO. Conversely, when no custom proposal is needed, Pyro offers automatic guide generation. section 13.3.3 section 2.2.5
