Kullback–Leibler divergence (KL divergence) measures how far a predicted probability distribution differs from a reference distribution. In a variational autoencoder (VAE), it compares each molecule’s latent distribution q(z|x) with a standard-normal prior p(z). section 10.3
A VAE encoder maps each molecular SMILES string to a probability cloud, commonly described by a mean μ and variance σ², rather than a single latent point. The KL term pulls these clouds toward a shared prior, helping turn disconnected latent islands into a continuous space that can be explored by sampling. section 10.3 section 10.5
Training combines reconstruction loss with KL regularization: L = reconstruction_loss + β KL. Reconstruction preserves the input SMILES, while β controls how strongly the latent distribution follows the prior. Excessive or poorly scheduled regularization can contribute to posterior collapse, where the decoder ignores the latent representation. section 10.3 section 10.4.5
Within molecular VAE training, KL divergence supports several practical operations:
- Regularize the latent distribution toward a standard normal prior. section 10.3
- Balance reconstruction accuracy against a smooth, sampleable latent space through the KL weight β. section 10.3
- Use cyclical annealing to vary the KL weight during training and reduce posterior collapse. section 10.4.5


















