Gating

As explained in Inside Deep Learning: Math, Algorithms, Models

A mechanism that modulates information flow by multiplying signals by gate values, typically in the range from zero to one.

Gating modulates information flow by multiplying a signal by a value between 0 and 1. A value of 0 blocks the signal, 1 passes it through, and intermediate values regulate its strength. section 6.6.2 section 6.8

An LSTM applies three gates to manage its long- and short-term states:

• Forget gate: controls which information leaves the context state Ct. section 6.6.2
• Input gate: controls what information enters Ct. section 6.6.2
• Output gate: controls how much of Ct contributes to the hidden state ht. section 6.6.2

A gated signal

Using the sigmoid activation, z = α · σ(·). When σ(·) is 0, z carries none of α; when it is 1, z carries all of α. Intermediate values partially transmit α. section 6.6.2

Figure 6.14 Diagram showing the detailed operations of an LSTM and how each gate is implemented and connected. The × indicates two values being multiplied together, and the + shows values being added together. The same color coding applies (red for the forget gate, blue for the input gate, and green for the output gate).

Use the detailed LSTM diagram to trace multiplication, addition, and the connections among the three gates, Ct, and ht. section 6.6.2

How Inside Deep Learning: Math, Algorithms, Models covers it

How deep it goes

Explains gate ranges, sigmoid and tanh roles, multiplication, and the three-gate LSTM arrangement.

Examples

Uses LSTM forget, input, and output gates plus the equation z = α · σ(·).

What you can do after

Evaluate how gates block, pass, or regulate signals within an LSTM.

How it connects

Connects sigmoid, tanh, LSTM, RNNs, hidden states, and context states.

liveBookmanning.com

Read Inside Deep Learning: Math, Algorithms, Models on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Sutskever's ListGating is a learned mechanism that controls how much newly computed information is combined with retained information.
• 

  liveBookmanning.com
  Natural Language Processing in Action, Second EditionA logic gate in a gated recurrent network is a learned signal-control mechanism that regulates how information passes into or out of the recurrent state.
• 

  liveBookmanning.com
  LLMs in ProductionA gating mechanism selectively controls how information enters, remains in, or leaves a recurrent model state.

Compare how 4 books explain gating →

Related concepts

• thresholduses · A threshold is a boundary value against which a quantity is compared to determine whether a condition is met or an action or decision is triggered.
• GRUpart of · A gated recurrent unit (GRU) is a recurrent neural-network architecture that uses learned gates to regulate information flow and update a hidden state across sequence steps.
• LSTMused by · An LSTM is a recurrent neural network architecture with gated memory mechanisms that control information flow and help retain or update information across sequences.
• neuronproduced by · A neuron is a computational unit in a neural network that transforms inputs, typically using learned parameters and an activation function, to produce an output.
• mixture of expertsused by · A mixture-of-experts model is a neural-network architecture with specialized expert subnetworks and a routing mechanism that selects or combines experts for each input.

This explainer was generated by Manning's knowledge graph from the text of Inside Deep Learning: Math, Algorithms, Models. The section links point to the parts of the book it draws on.