VAE-CYC is a variational molecular autoencoder that converts SMILES strings into continuous latent vectors and decodes them back into molecular sequences. Its purpose is to create a chemical map that optimization methods can navigate. section 10.2
The architecture combines the following stages:
- Tokenize SMILES with SmilesPE and summarize the sequence using a bidirectional GRU encoder. section 10.4.5
- Project the encoder output into latent mean and log-variance parameters, then sample z with the reparameterization trick. section 10.4.5
- Decode z autoregressively with a unidirectional GRU, generating one token at a time. section 10.4.5
- Optimize reconstruction loss together with KL divergence, while cyclically changing the KL weight. section 10.4.5 section 10.3
Cyclical annealing addresses posterior collapse, where an autoregressive decoder reconstructs sequences without using z. The KL weight repeatedly rises from zero to its maximum and resets, reopening the latent space for refinement. section 10.3
Targeted molecular generation
A trained VAE-CYC samples latent vectors from the prior and decodes them into molecules. Within MOLRL, an RL agent navigates this latent space toward regions associated with objectives such as activity and low toxicity; generated molecules are then assessed for validity, uniqueness, novelty, and diversity. section 10.1.3 section 10.4.5
Listing 10.13 Defining the Model Architecture
class VAECYC(
AbstractMolecularAutoencoder
): #A
def __init__(
self, vocab: Vocab, latent_dim: int = 64, max_length: int = 100,
embed_dim: int = 128, hidden_dim: int = 512,
encoder_num_layers: int = 2, encoder_bidirectional: bool = True,
encoder_dropout: float = 0.2, decoder_num_layers: int = 4,
decoder_dropout: float = 0.2, max_kl_weight: float = 0.6,
n_cycles: int = 20, cycle_ratio: float = 0.7,
word_dropout: float = 0.2, free_bits: float = 0.25,
max_epochs: int = 10
): #B
super().__init__()
self.vocab = vocab
self.latent_dim = latent_dim #C
self.embedding = nn.Embedding(
num_embeddings=self.vocab_size,
embedding_dim=self.embed_dim,
padding_idx=vocab.pad_idx #D
)
self.encoder = nn.GRU(
input_size=self.embed_dim,
hidden_size=self.hidden_dim,
num_layers=self.encoder_num_layers,
batch_first=True, #E
bidirectional=self.encoder_bidirectional, #F
dropout= \
self.encoder_dropout if self.encoder_num_layers > 1 else 0
)
encoder_output_dim = self.hidden_dim * (
2 if self.encoder_bidirectional else 1
) #G
self.fc_mu = nn.Linear(
encoder_output_dim, self.latent_dim
) #H
self.fc_logvar = nn.Linear(
encoder_output_dim, self.latent_dim
) #I
self.decoder = nn.GRU(
input_size=self.embed_dim + self.latent_dim, #J
hidden_size=self.hidden_dim,
num_layers=self.decoder_num_layers,
batch_first=True,
bidirectional=False, #K
dropout= \
self.decoder_dropout if self.decoder_num_layers > 1 else 0
)
self.latent_to_hidden = nn.Linear(
self.latent_dim,
self.hidden_dim * self.decoder_num_layers #L
)
self.output_projection = nn.Linear(
self.hidden_dim, self.vocab_size
) #M
self.register_buffer(
'step_count', torch.tensor(0, dtype=torch.long)
) #N
self._initialize_weights() #OUse the architecture listing to inspect the PyTorch model configuration, including latent dimensions, GRU layers, dropout, and cyclical-annealing parameters. section 10.4.5
