liveBook · Manning

VAE-CYC

As explained in Build AI Drug Discovery Pipelines

VAE-CYC is a variational molecular autoencoder architecture that combines sequence encoding, latent-variable sampling, autoregressive decoding, and cyclical KL regularization.

VAE-CYC is a variational molecular autoencoder that converts SMILES strings into continuous latent vectors and decodes them back into molecular sequences. Its purpose is to create a chemical map that optimization methods can navigate. section 10.2

The architecture combines the following stages:

  • Tokenize SMILES with SmilesPE and summarize the sequence using a bidirectional GRU encoder. section 10.4.5
  • Project the encoder output into latent mean and log-variance parameters, then sample z with the reparameterization trick. section 10.4.5
  • Decode z autoregressively with a unidirectional GRU, generating one token at a time. section 10.4.5
  • Optimize reconstruction loss together with KL divergence, while cyclically changing the KL weight. section 10.4.5 section 10.3

Cyclical annealing addresses posterior collapse, where an autoregressive decoder reconstructs sequences without using z. The KL weight repeatedly rises from zero to its maximum and resets, reopening the latent space for refinement. section 10.3

Targeted molecular generation

A trained VAE-CYC samples latent vectors from the prior and decodes them into molecules. Within MOLRL, an RL agent navigates this latent space toward regions associated with objectives such as activity and low toxicity; generated molecules are then assessed for validity, uniqueness, novelty, and diversity. section 10.1.3 section 10.4.5

Listing 10.13 Defining the Model Architecture
class VAECYC(
    AbstractMolecularAutoencoder
):   #A
    def __init__(
        self, vocab: Vocab, latent_dim: int = 64, max_length: int = 100,
        embed_dim: int = 128, hidden_dim: int = 512,
        encoder_num_layers: int = 2, encoder_bidirectional: bool = True,          
        encoder_dropout: float = 0.2, decoder_num_layers: int = 4, 
        decoder_dropout: float = 0.2, max_kl_weight: float = 0.6, 
        n_cycles: int = 20, cycle_ratio: float = 0.7, 
        word_dropout: float = 0.2, free_bits: float = 0.25,
        max_epochs: int = 10
    ):   #B
        
        super().__init__()
        
        self.vocab = vocab 
        self.latent_dim = latent_dim   #C
                
        self.embedding = nn.Embedding(
            num_embeddings=self.vocab_size, 
            embedding_dim=self.embed_dim,
            padding_idx=vocab.pad_idx   #D
        )
        
        self.encoder = nn.GRU(
            input_size=self.embed_dim, 
            hidden_size=self.hidden_dim,
            num_layers=self.encoder_num_layers,
            batch_first=True,   #E
            bidirectional=self.encoder_bidirectional,   #F
            dropout= \
                self.encoder_dropout if self.encoder_num_layers > 1 else 0
        )
        
        encoder_output_dim = self.hidden_dim * (
            2 if self.encoder_bidirectional else 1
        )   #G
        
        self.fc_mu = nn.Linear(
            encoder_output_dim, self.latent_dim
        )   #H
        
        self.fc_logvar = nn.Linear(
            encoder_output_dim, self.latent_dim
        )   #I
        
        self.decoder = nn.GRU(
            input_size=self.embed_dim + self.latent_dim,   #J
            hidden_size=self.hidden_dim,
            num_layers=self.decoder_num_layers,
            batch_first=True,
            bidirectional=False,   #K
            dropout= \
                self.decoder_dropout if self.decoder_num_layers > 1 else 0
        )
        
        self.latent_to_hidden = nn.Linear(
            self.latent_dim, 
            self.hidden_dim * self.decoder_num_layers   #L
        )
        
        self.output_projection = nn.Linear(
            self.hidden_dim, self.vocab_size
        )   #M
        
        self.register_buffer(
            'step_count', torch.tensor(0, dtype=torch.long)
        )   #N
        
        self._initialize_weights()   #O

Use the architecture listing to inspect the PyTorch model configuration, including latent dimensions, GRU layers, dropout, and cyclical-annealing parameters. section 10.4.5

How Build AI Drug Discovery Pipelines covers it

How deep it goes
Explains the full encoding, sampling, decoding, training, annealing, generation, and evaluation workflow, with implementation details for the model class.
Examples
Uses SMILES sequences, prior-sampled molecules, activity and toxicity regions, and validity, uniqueness, novelty, and diversity metrics.
What it assumes
Starts from SMILES strings, standard autoencoders, and recurrent neural networks before introducing variational sampling and cyclical regularization.
What you can do after
Build it from scratch by assembling a PyTorch molecular autoencoder and train it for latent-space molecular generation.
How it connects
Connects SMILES, GRUs, variational autoencoders, latent spaces, KL divergence, posterior collapse, and reinforcement learning.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Build AI Drug Discovery Pipelines. The section links point to the parts of the book it draws on.