A reset gate is a gated recurrent mechanism that decides which portions of the previous hidden state are no longer relevant to the current output. It belongs to a gated recurrent unit (GRU), where gates are placed inside the recurrent unit rather than added as an ordinary layer. section 8.4.2
Within a GRU, the gates divide the recurrent decision into complementary roles:
- The reset gate blocks selected parts of the hidden layer when they are irrelevant to the current output. section 8.4.2
- The update gate determines which parts of the hidden layer should matter at the current time. section 8.4.2
Gate computation
A reset vector can be computed with a sigmoid: r = sigmoid(W_i2r·x + b_i2r + W_h2r·h + b_h2r). The resulting r scales the recurrent hidden-state contribution inside the tanh computation for n, allowing selected information to be reduced before the new activation is formed.
