Encoding arithmetic manipulates a variational autoencoder’s latent representations by adding and subtracting averaged encodings, then decoding the result into a new image. A regularized latent space makes related features sufficiently continuous and interpretable for this operation. section 7.4.4
Removing glasses through latent arithmetic
Let z1 represent men with glasses, z2 women with glasses, and z3 women without glasses. Compute z = z1 − z2 + z3, then decode z to target a man-without-glasses image.
Encoding arithmetic differs from encoding interpolation: arithmetic targets a feature transformation, whereas interpolation gradually blends two encodings by changing a weight from 0 to 1.
