Generation models a process for producing observations by representing a joint or conditional probability distribution. In a structural causal model, latent exogenous variables represent unmodeled causes, while assignment functions generate endogenous variables from their parents. section 6.1.6
Causal generation extends ordinary prediction by preserving a shared latent explanation of an observed case. Counterfactual generation first infers exogenous variables from factual observations, intervenes on the desired condition, and predicts the resulting outcome. section 9.1 section 9.5
A practical generation workflow follows three steps:
- Abduction: infer the distribution of latent exogenous variables from observed endogenous variables. section 9.5
- Action: implement the hypothetical condition as an ideal intervention or graph surgery. section 9.1
- Prediction: sample hypothetical outcomes using the intervention and the latent distribution learned from the actual world. section 9.3.4
Semantic image editing
Given an image of a pirate-captain interpretation of Harriet Tubman, infer its latent representation, intervene to remove the concept of glasses, and generate a revised image. A causal representation aims to preserve features unaffected by that intervention while propagating downstream changes. section 9.4
Figure 9.16 Given the generated image on the left, the user might prompt the generative AI with the counterfactual question, “What would this image look like without the glasses?” They would expect something like the image on the right, where conceptual elements of the image not causally downstream of glasses removal should be unaffected.
Compare the original generated image with its counterfactual edit and inspect which conceptual elements should remain unchanged. section 9.4
