A digital image is a grid of numerical measurements. In grayscale datasets such as MNIST and TMNIST, each image is a 28 × 28 matrix, or 784 pixel features, whose values encode color intensity. section 5.1.2
Pixel values support statistical prediction, but they are often too low-level for causal questions. A useful image model instead represents higher-level causes—such as shape, scale, rotation, and position—and treats the image as their generated outcome. section 5.1.2 section 9.4.1
Changing a shape
For a dSprites image containing a square at a particular scale, rotation, and position, replace the shape factor with heart and decode the intervened factors with the inferred image variation. section 9.4.1
Figure 5.3 An MNIST image of “6” (left) and a TMNIST image of “7”. In their raw form, these are 28 × 28 matrices of numeric values corresponding to grayscale values.
Compare rendered MNIST and TMNIST digits with their underlying 28 × 28 grayscale matrices. section 5.1.2
Figure 9.20 The parallel world graph implied by the question “Given the image, what would it look like if the shape were a heart?”
Trace the parallel-world graph for asking how an observed image would look under a changed shape. section 9.4.1
