Carry state

As explained in Inside Deep Learning: Math, Algorithms, Models

The LSTM memory state that carries longer-term information separately from the hidden state.

Carry state, called the context state Cₜ in an LSTM, preserves valuable information for later time steps. It focuses on longer-term information, while the hidden state hₜ handles short-term work and complex functions. section 6.6.2

LSTM gates regulate how information enters, remains in, and leaves the carry state:

• The forget gate removes information from Cₜ. section 6.6.2
• The input gate controls what new information enters Cₜ. section 6.6.2
• The output gate controls how much of Cₜ contributes to the hidden state hₜ. section 6.6.2
• Sigmoid gates produce values from 0 to 1: zero blocks information, one passes it through, and intermediate values regulate flow. section 6.6.2

Figure 6.13 The strategy of an LSTM involves three gates that operate sequentially and allow interaction between the short term h_t and the long term C_t. The forget gate is red, the input gate is blue, and the output gate is green.

Use the gate diagram to trace sequential interaction between the carry state Cₜ and hidden state hₜ. section 6.6.2

How Inside Deep Learning: Math, Algorithms, Models covers it

How deep it goes

Explains the state’s role and its three gates, stopping short of complete update equations or implementation code.

Examples

Uses forget, input, and output gates plus diagrams showing multiplication, addition, and state interaction.

What you can do after

Understand how LSTM gates preserve, discard, and expose longer-term information through the carry state.

How it connects

Connects carry state to LSTMs, hidden states, gating mechanisms, sigmoid, tanh, and residual networks.

liveBookmanning.com

Read Inside Deep Learning: Math, Algorithms, Models on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Sutskever's ListA cell state is the long-term memory pathway in an LSTM that carries information across time steps with selectively gated updates.
• 

  liveBookmanning.com
  Natural Language Processing in Action, Second EditionA cell state is the persistent recurrent representation maintained by an LSTM across sequence positions.
• 

  liveBookmanning.com
  Deep Learning with R, Third EditionA carry state is an information stream in an LSTM that transports a representation across sequence timesteps for later modification or use.
• 

  liveBookmanning.com
  Deep Learning with Python, Third EditionThe LSTM internal state that transports information across timesteps to preserve long-term context.

Compare how 5 books explain carry state →

Related concepts

• LSTMpart of · An LSTM is a recurrent neural network architecture with gated memory mechanisms that control information flow and help retain or update information across sequences.

This explainer was generated by Manning's knowledge graph from the text of Inside Deep Learning: Math, Algorithms, Models. The section links point to the parts of the book it draws on.