Gating
Gating is the selective regulation of information flow within a computational system, controlling what information is passed, retained, combined, or suppressed.
Sutskever's List
Gating is a learned mechanism that controls how much newly computed information is combined with retained information.
- How it is used
- Uses learned gates to control information retention, updates, and output in recurrent memory.
- How deep it goes
- Explains LSTM gate roles and elementwise filtering, then relates gating to RMC memory updates.
- Examples
- Uses an LSTM cell diagram and an RMC task comparing vectors by distance from a reference.
- What you can do after
- Understand how gates regulate memory retention, updating, and exposure in recurrent models.
Natural Language Processing in Action, Second Edition
A logic gate in a gated recurrent network is a learned signal-control mechanism that regulates how information passes into or out of the recurrent state.
- How it is used
- Explains how GRUs and LSTMs preserve useful information across long token sequences.
- How deep it goes
- Works from an if-statement analogy through learned thresholds and gate equations, while omitting full training procedures.
- Examples
- Uses long-text token memory, GRU gate equations, LSTM state-update equations, and architecture figures as concrete examples.
- What you can do after
- Build GRU and LSTM gate calculations using sigmoid, tanh, learned weights, biases, and recurrent states.
Read how Natural Language Processing in Action, Second Edition explains gating →
LLMs in Production
A gating mechanism selectively controls how information enters, remains in, or leaves a recurrent model state.
- How it is used
- Uses gates to preserve useful sequence information and select experts during recurrent modeling or mixture-of-experts fine-tuning.
- How deep it goes
- Explains gating through LSTM memory cells, sequence-length effects, bidirectionality, and an implementation comparison, but not individual gate equations.
- Examples
- Uses Twitter sentiment analysis, variable-length padded sequences, word embeddings, and an RNN-versus-LSTM training comparison.
- What you can do after
- Evaluate/understand how LSTM gating improves long-sequence processing and compare it with an RNN in a sentiment workflow.
Inside Deep Learning: Math, Algorithms, Models
A mechanism that modulates information flow by multiplying signals by gate values, typically in the range from zero to one.
- How it is used
- Regulates information flow in LSTMs to preserve useful long-term information while producing short-term hidden states.
- How deep it goes
- Explains gate ranges, sigmoid and tanh roles, multiplication, and the three-gate LSTM arrangement.
- Examples
- Uses LSTM forget, input, and output gates plus the equation z = α · σ(·).
- What you can do after
- Evaluate how gates block, pass, or regulate signals within an LSTM.
Read how Inside Deep Learning: Math, Algorithms, Models explains gating →
Related concepts
- thresholduses · A threshold is a boundary value against which a quantity is compared to determine a decision, classification, or action.
- GRUpart of · A gated recurrent unit (GRU) is a recurrent neural-network architecture that uses learned gates to regulate information flow and update a hidden state across sequence steps.
- LSTMused by · An LSTM is a recurrent neural network architecture with gated memory mechanisms that control information flow and help retain or update information across sequences.
- neuronproduced by · A neuron is a computational unit in a neural network that transforms inputs, typically using learned parameters and an activation function, to produce an output.
- mixture of expertsused by · A mixture-of-experts model is a neural-network architecture with specialized expert subnetworks and a routing mechanism that selects or combines experts for each input.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.