liveBook · Manning

GRU

As explained in Machines that Think

A recurrent neural network architecture that uses gates to control the retention and updating of sequence information.

A Gated Recurrent Unit (GRU) is a recurrent neural network architecture designed to retain useful information across a sequence. It addresses a weakness of basic recurrent networks: signals passed from one step to the next can fade or grow uncontrollably, making distant relationships difficult to capture. section 3.4.4

GRUs use trainable gates to regulate information flow over time. Learned weights and activation functions produce values that control what information is retained, updated, or discarded. A GRU is a streamlined alternative to a Long Short-Term Memory network (LSTM), combining some of the LSTM’s separate memory mechanisms into fewer components. section 3.4.4

A long-range dependency

In “The book that I borrowed last week was fascinating,” interpreting “was fascinating” depends on retaining information about “the book” across the intervening words. Gated recurrent architectures are designed to preserve relevant context across such gaps while filtering less useful detail. section 3.4.4

Selective memory has limits. GRUs process language one token at a time, so earlier information is accessible only through the evolving state carried forward. This sequential structure limits parallelism, while compression into a shared pathway can cause distant dependencies to interfere with one another. section 3.5.3

How Machines that Think covers it

How deep it goes
Explains learned gating and contrasts GRUs with LSTMs and basic recurrent networks, without detailing individual GRU gates.
Examples
Illustrates long-range context with a sentence connecting “the book” to “was fascinating.”
What you can do after
Evaluate GRUs as sequence models by weighing selective memory against sequential processing and state compression.
How it connects
Connects GRUs to recurrent neural networks, LSTMs, hidden state, and long-range dependencies.

Also explained in

Compare how 10 books explain GRU →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Machines that Think. The section links point to the parts of the book it draws on.