A decoder turns an internal representation into an output. In a molecular autoencoder, it takes a compressed latent vector and tries to reconstruct the original molecule’s SMILES string, a text representation of its structure. Training minimizes the difference between the input and reconstruction. section 10.2.2
Decoders can produce outputs in different ways. A basic molecular autoencoder expands the latent vector through neural-network layers and outputs scores for each token position. A sequence-to-sequence decoder instead generates a SMILES string one token at a time, using its evolving hidden state to carry information forward. section 10.2.3 section 10.4.1
Listing 10.4 Basic Autoencoder Architecture: Decoder
decoder_layers = []
input_dim = latent_dim #A
for hidden_dim in reversed(hidden_dims): #B
decoder_layers.extend([
nn.Linear(input_dim, hidden_dim),
nn.ReLU(),
nn.BatchNorm1d(hidden_dim),
nn.Dropout(0.1)
])
input_dim = hidden_dim
decoder_layers.extend([ #C
nn.Linear(input_dim, vocab_size * max_length),
nn.Unflatten(1, (max_length, vocab_size)) #D
])
self.decoder = nn.Sequential(*decoder_layers)
self._initialize_weights() #EInspect a PyTorch decoder that expands a latent vector into token scores for a fixed maximum sequence length. section 10.2.3
In a molecular variational autoencoder (VAE), a bidirectional GRU encoder produces parameters for a latent distribution, from which a vector is sampled. A unidirectional GRU decoder uses that vector to generate the output SMILES. Its initial hidden state comes from the vector, which is also joined to token embeddings at each step. section 10.4.3
Listing 10.10 Unidirectional Decoder with Latent Concatenation
def decode(
self, x: torch.Tensor, z: torch.Tensor
) -> torch.Tensor: #A
batch_size, seq_len = x.size()
h_init = self.latent_to_hidden(z) #B
h = h_init.view(batch_size, self.decoder_num_layers, self.hidden_dim)
h = h.transpose(0, 1).contiguous() #C
embedded = self.embedding(x) #D
z_expanded = z.unsqueeze(1).expand(
batch_size, seq_len, self.latent_dim
) #E
decoder_input = torch.cat(
[embedded, z_expanded], dim=2
) #F
output, _ = self.decoder(decoder_input, h) #G
logits = self.output_projection(output) #H
return logits Follow the decoder’s latent-to-hidden projection, token embedding, latent concatenation, GRU, and output projection. section 10.4.3
During training, teacher forcing supplies ground-truth tokens to the recurrent decoder. During generation, it starts with a start token and feeds its predictions back as inputs until it produces an end token. The basic autoencoder’s reconstruction loss uses cross-entropy and ignores padding tokens. section 10.4.3 section 10.4.1 section 10.2.3
Reconstruction alone is not enough for molecular design. A decoded SMILES must also be chemically valid: RDKit should be able to parse it into a molecule. Invalid outputs cannot provide meaningful molecular-property feedback to a reinforcement-learning agent. Structural similarity can help assess how faithfully an encoded molecule is reconstructed. section 10.2.1
A standard Transformer also has a decoder, but it uses cross-attention to consult the encoder’s token representations while generating a target sequence. An encoder-only Transformer omits the decoder when the goal is to represent an input, rather than generate a new sequence. section 12.3.2








