A gated recurrent unit (GRU) is a recurrent neural-network unit that uses learned gates to control how information passes from one sequence step to the next. Its hidden state acts as a running memory. This helps address vanishing gradients, which can make simple recurrent networks struggle to learn dependencies across long sequences. section 10.4.2
Remembering molecular syntax
A SMILES string represents a molecule as a sequence. In the benzene string c1ccccc1, the two 1s mark a ring closure; a generator must keep track of the opening marker until it produces the matching closure. A GRU helps retain such context, although it does not guarantee a valid molecule. section 10.4.2
At each step, the GRU combines the current input with its previous hidden state:
- The reset gate controls how much of the previous state contributes to a candidate new state. section 10.4.2
- The update gate controls the blend of the previous state and the candidate state to produce the next hidden state. section 10.4.2
Figure 10.6 Internal architecture of a Gated Recurrent Unit (GRU) cell. The diagram illustrates the information flow through a GRU cell, showing the two key gates (reset gate rt and update gate zt) that control memory retention and updates. The reset gate determines how much previous hidden state information to retain when computing the candidate hidden state h̃t, while the update gate controls the blend between the previous hidden state ht-1 and the new candidate state to produce the final output ht. The mathematical equations on the right define the computations for each component, with σ representing the sigmoid activation function and ⊗ denoting elementwise multiplication.

Follow the reset and update gates through a single cell, from input and previous state to the next hidden state. section 10.4.2
This selective memory costs more computation than a simple RNN. Counting only internal cell computations, a simple RNN uses two weight matrices, while a GRU uses six: two for each gate and two for the candidate state. That means roughly three times as many matrix multiplications per step, with higher training time and memory requirements. Both models can also use a separate hidden-to-output projection. section 10.4.2
For molecular generation, a sequence-to-sequence variational autoencoder can use two GRUs with different jobs. A bidirectional encoder reads the complete input SMILES in both directions and summarizes it to produce latent-distribution parameters. A unidirectional decoder uses a sampled latent vector to generate an output SMILES one token at a time. section 10.4.3 section 10.4.5
Listing 10.10 Unidirectional Decoder with Latent Concatenation
def decode(
self, x: torch.Tensor, z: torch.Tensor
) -> torch.Tensor: #A
batch_size, seq_len = x.size()
h_init = self.latent_to_hidden(z) #B
h = h_init.view(batch_size, self.decoder_num_layers, self.hidden_dim)
h = h.transpose(0, 1).contiguous() #C
embedded = self.embedding(x) #D
z_expanded = z.unsqueeze(1).expand(
batch_size, seq_len, self.latent_dim
) #E
decoder_input = torch.cat(
[embedded, z_expanded], dim=2
) #F
output, _ = self.decoder(decoder_input, h) #G
logits = self.output_projection(output) #H
return logits The PyTorch decoder projects the latent vector into initial hidden states, concatenates it with token embeddings, and projects GRU outputs to logits. section 10.4.3
After generation, evaluate molecular validity alongside measures such as uniqueness, novelty, and diversity. Better sequence memory is useful, but generation quality still needs to be checked. section 10.4.5 section 10.4.2









