Posterior collapse is a variational autoencoder (VAE) failure mode: the decoder ignores the latent representation, while the learned posterior approaches the prior. In molecular generation, this means the compressed representation may carry too little useful information about the input molecule. section 10.3 section 10.5
A molecular VAE balances two objectives. Reconstruction loss asks the decoder to rebuild an input SMILES string from a sampled latent vector. KL divergence pulls the encoder’s distribution toward a fixed prior; its weight, β, controls its contribution to the loss. Collapse undermines the intended use of the latent space for generation. section 10.3 section 10.5
An autoregressive decoder generates tokens one at a time and can learn to rely on the sequence rather than the latent vector. Long character-level sequences make the encoder’s compression task harder: individual characters may carry little chemical meaning. section 10.4.4 section 10.4.5
From characters to chemical motifs
At the character level, the model must learn that the sequence c1ccccc1 represents a benzene ring. Smiles Pair Encoding can combine recurring patterns into meaningful tokens, producing shorter sequences and a more structured latent representation that is less prone to collapse. section 10.4.4 section 10.5
Two complementary ways to reduce collapse risk are:
- Use cyclical annealing to adjust the KL-divergence weight during training, encouraging the decoder to use the latent vector. section 10.4.5 section 10.5
- Use chemically meaningful tokens, such as those learned by Smiles Pair Encoding, to make molecular sequences easier to compress. section 10.4.4
