An update gate is the GRU mechanism that controls how much of a new candidate hidden state replaces or supplements the previous hidden state. It acts as a learned filter for retaining sequence information. section 10.4.2
At each sequence step, the GRU:
- Computes z_t from the current input and previous hidden state using a sigmoid. section 10.4.2
- Blends the previous hidden state with the candidate state: h_t = (1 − z_t) ⊙ h_{t−1} + z_t ⊙ h̃_t. section 10.4.2
- Retains information when the gate favors the previous state and incorporates new information when it favors the candidate. section 10.4.2
Molecular sequence generation
When processing a SMILES string, the update gate helps preserve earlier structural information, such as an opening parenthesis or ring start, until the matching symbol appears later. section 10.4.2
Figure 10.6 Internal architecture of a Gated Recurrent Unit (GRU) cell. The diagram illustrates the information flow through a GRU cell, showing the two key gates (reset gate rt and update gate zt) that control memory retention and updates. The reset gate determines how much previous hidden state information to retain when computing the candidate hidden state h̃t, while the update gate controls the blend between the previous hidden state ht-1 and the new candidate state to produce the final output ht. The mathematical equations on the right define the computations for each component, with σ representing the sigmoid activation function and ⊗ denoting elementwise multiplication.

Inspect the GRU cell diagram to follow the update gate’s blend of the previous hidden state and candidate state. section 10.4.2

