liveBook · Manning

GRU

As explained in Build AI Drug Discovery Pipelines

A gated recurrent unit is a recurrent neural-network unit that uses learned gates to control information retained and propagated across sequence steps.

A gated recurrent unit (GRU) is a recurrent neural-network unit that uses learned gates to control how information passes from one sequence step to the next. Its hidden state acts as a running memory. This helps address vanishing gradients, which can make simple recurrent networks struggle to learn dependencies across long sequences. section 10.4.2

Remembering molecular syntax

A SMILES string represents a molecule as a sequence. In the benzene string c1ccccc1, the two 1s mark a ring closure; a generator must keep track of the opening marker until it produces the matching closure. A GRU helps retain such context, although it does not guarantee a valid molecule. section 10.4.2

At each step, the GRU combines the current input with its previous hidden state:

  • The reset gate controls how much of the previous state contributes to a candidate new state. section 10.4.2
  • The update gate controls the blend of the previous state and the candidate state to produce the next hidden state. section 10.4.2
Figure 10.6 Internal architecture of a Gated Recurrent Unit (GRU) cell. The diagram illustrates the information flow through a GRU cell, showing the two key gates (reset gate rt and update gate zt) that control memory retention and updates. The reset gate determines how much previous hidden state information to retain when computing the candidate hidden state h̃t, while the update gate controls the blend between the previous hidden state ht-1 and the new candidate state to produce the final output ht. The mathematical equations on the right define the computations for each component, with σ representing the sigmoid activation function and ⊗ denoting elementwise multiplication.

Follow the reset and update gates through a single cell, from input and previous state to the next hidden state. section 10.4.2

This selective memory costs more computation than a simple RNN. Counting only internal cell computations, a simple RNN uses two weight matrices, while a GRU uses six: two for each gate and two for the candidate state. That means roughly three times as many matrix multiplications per step, with higher training time and memory requirements. Both models can also use a separate hidden-to-output projection. section 10.4.2

For molecular generation, a sequence-to-sequence variational autoencoder can use two GRUs with different jobs. A bidirectional encoder reads the complete input SMILES in both directions and summarizes it to produce latent-distribution parameters. A unidirectional decoder uses a sampled latent vector to generate an output SMILES one token at a time. section 10.4.3 section 10.4.5

Listing 10.10 Unidirectional Decoder with Latent Concatenation
def decode(
    self, x: torch.Tensor, z: torch.Tensor
) -> torch.Tensor:    #A
    batch_size, seq_len = x.size()
    
    h_init = self.latent_to_hidden(z)    #B
    h = h_init.view(batch_size, self.decoder_num_layers, self.hidden_dim)
    h = h.transpose(0, 1).contiguous()    #C
    
    embedded = self.embedding(x)    #D
    
    z_expanded = z.unsqueeze(1).expand(
        batch_size, seq_len, self.latent_dim
    )    #E
    
    decoder_input = torch.cat(
        [embedded, z_expanded], dim=2
    )    #F
    output, _ = self.decoder(decoder_input, h)    #G
    logits = self.output_projection(output)    #H
    return logits           

The PyTorch decoder projects the latent vector into initial hidden states, concatenates it with token embeddings, and projects GRU outputs to logits. section 10.4.3

After generation, evaluate molecular validity alongside measures such as uniqueness, novelty, and diversity. Better sequence memory is useful, but generation quality still needs to be checked. section 10.4.5 section 10.4.2

How Build AI Drug Discovery Pipelines covers it

How deep it goes
Explains gate computations and cell costs, then implements GRU-based encoder and decoder components in a molecular variational autoencoder.
Examples
Uses benzene ring closure and a molecular encoder-decoder that reconstructs SMILES from sampled latent vectors.
What you can do after
Apply it via PyTorch to encode and decode molecular sequences in a variational autoencoder.
How it connects
Connects GRUs to vanishing gradients, recurrent neural networks, sequence-to-sequence models, and variational autoencoders.

Also explained in

Compare how 10 books explain GRU →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Build AI Drug Discovery Pipelines. The section links point to the parts of the book it draws on.