How deeply does Sutskever's List cover GRU?

Explains bidirectional encoding and decoder inputs but stops short of detailing individual gate equations.

GRU

A gated recurrent unit is a recurrent neural-network unit that uses learned gates to regulate state updates.

A gated recurrent unit (GRU) is a recurrent neural-network unit whose learned gates regulate updates to its hidden state. In Bahdanau, Cho, and Bengio’s RNNsearch architecture, bidirectional GRUs encode a sentence while a GRU decoder generates its translation. section 5.3

The encoder runs one GRU forward and another backward, joining their outputs into word-level annotations. The decoder uses a 1,000-unit GRU that combines its previous state, the last generated word, and an attention-derived context vector. section 5.3

GRUs in neural machine translation

On the WMT English–French benchmark, RNNsearch paired GRU encoding and decoding with attention, outperforming fixed-vector encoder-decoder baselines as sentence-length limits increased. section 5.3

How Sutskever's List covers it

liveBook manning.com

Read Sutskever's List on liveBook Get the book at manning.com

Also explained in

liveBook manning.com

Inside Deep Learning: Math, Algorithms, ModelsA gated recurrent neural network architecture that updates a hidden state over sequence steps with reset and update gates.
liveBook manning.com

Graph Neural Networks in ActionA recurrent neural-network architecture that uses reset and update gates to control information flow through its recurrent state.
liveBook manning.com

Deep Learning with Python, Third EditionA recurrent neural network unit designed to process sequences with gated state updates.
liveBook manning.com

Deep Learning with R, Third EditionA gated recurrent unit (GRU) is a recurrent neural-network architecture that models sequence dependencies with update and reset mechanisms.
liveBook manning.com

Natural Language Processing in Action, Second EditionA gated recurrent unit (GRU) is a recurrent neural-network architecture that uses reset and update gates to control information flow and hidden-state updates.
liveBook manning.com

Grokking Machine LearningA gated recurrent unit is a recurrent neural-network unit that uses gates to control the retention and update of sequence state.

Compare how 7 books explain GRU →

Related concepts

gated updatehas part · A gated update is a state or representation update in which learned gates control how new information is combined with or replaces existing information.
reset gatehas part · A reset gate is a GRU mechanism that controls how much of the previous hidden state contributes to computing a candidate state.
vanishing gradient problemaddresses · The vanishing-gradient problem is a neural-network training difficulty in which gradients shrink toward zero as they propagate backward, making updates to earlier layers or distant sequence steps ineffective.
LSTMalternative to · An LSTM is a recurrent neural network architecture with gated memory mechanisms that control information flow and help retain or update information across sequences.
recurrent neural networkaddresses · A recurrent neural network is a neural network that processes sequences by updating an internal state across successive inputs, allowing earlier inputs to influence later computations.
bidirectional rnncontrasts with · A bidirectional recurrent neural network processes a sequence in forward and reverse directions and combines the resulting representations.

This explainer was generated by Manning's knowledge graph from the text of Sutskever's List. The section links point to the parts of the book it draws on.