A Gated Recurrent Unit (GRU) is a recurrent neural network architecture designed to retain useful information across a sequence. It addresses a weakness of basic recurrent networks: signals passed from one step to the next can fade or grow uncontrollably, making distant relationships difficult to capture. section 3.4.4
GRUs use trainable gates to regulate information flow over time. Learned weights and activation functions produce values that control what information is retained, updated, or discarded. A GRU is a streamlined alternative to a Long Short-Term Memory network (LSTM), combining some of the LSTM’s separate memory mechanisms into fewer components. section 3.4.4
A long-range dependency
In “The book that I borrowed last week was fascinating,” interpreting “was fascinating” depends on retaining information about “the book” across the intervening words. Gated recurrent architectures are designed to preserve relevant context across such gaps while filtering less useful detail. section 3.4.4
Selective memory has limits. GRUs process language one token at a time, so earlier information is accessible only through the evolving state carried forward. This sequential structure limits parallelism, while compression into a shared pathway can cause distant dependencies to interfere with one another. section 3.5.3









