liveBook · Manning

Posterior collapse

As explained in Build AI Drug Discovery Pipelines

Posterior collapse is a variational autoencoder failure mode in which the decoder ignores the latent representation and the learned posterior approaches the prior.

Posterior collapse is a variational autoencoder (VAE) failure mode: the decoder ignores the latent representation, while the learned posterior approaches the prior. In molecular generation, this means the compressed representation may carry too little useful information about the input molecule. section 10.3 section 10.5

A molecular VAE balances two objectives. Reconstruction loss asks the decoder to rebuild an input SMILES string from a sampled latent vector. KL divergence pulls the encoder’s distribution toward a fixed prior; its weight, β, controls its contribution to the loss. Collapse undermines the intended use of the latent space for generation. section 10.3 section 10.5

An autoregressive decoder generates tokens one at a time and can learn to rely on the sequence rather than the latent vector. Long character-level sequences make the encoder’s compression task harder: individual characters may carry little chemical meaning. section 10.4.4 section 10.4.5

From characters to chemical motifs

At the character level, the model must learn that the sequence c1ccccc1 represents a benzene ring. Smiles Pair Encoding can combine recurring patterns into meaningful tokens, producing shorter sequences and a more structured latent representation that is less prone to collapse. section 10.4.4 section 10.5

Two complementary ways to reduce collapse risk are:

  • Use cyclical annealing to adjust the KL-divergence weight during training, encouraging the decoder to use the latent vector. section 10.4.5 section 10.5
  • Use chemically meaningful tokens, such as those learned by Smiles Pair Encoding, to make molecular sequences easier to compress. section 10.4.4

How Build AI Drug Discovery Pipelines covers it

How deep it goes
Explains the reconstruction–KL trade-off and mitigation strategies, while deferring longer probabilistic derivations and complete training code.
Examples
Uses character-level benzene notation and Smiles Pair Encoding to illustrate how molecular tokenization affects collapse risk.
What you can do after
Understand how KL-weight scheduling and chemically meaningful tokens can reduce collapse risk in a molecular VAE.
How it connects
Connects posterior collapse to KL divergence, cyclical annealing, autoregressive decoding, inductive bias, and tokenization.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Build AI Drug Discovery Pipelines. The section links point to the parts of the book it draws on.