Carry state

As explained in Sutskever's List

A cell state is the long-term memory pathway in an LSTM that carries information across time steps with selectively gated updates.

Carry state is the LSTM’s long-term memory pathway: a cell state moves information across time steps while gates selectively retain, add, or remove it. This nearly unchanged path helps preserve information and ease gradient flow over long sequences, unlike the hidden state, which responds more actively to current inputs. section 4.2

The conveyor belt

An LSTM cell can be pictured as a conveyor belt carrying information through a sequence. The forget gate scales old information, the input gate and candidate update add new information, and the output gate controls what becomes visible through the hidden state. section 4.2

Figure 4.2 A single LSTM cell. The inputs are the current vector X_t, the previous hidden state h_t–1, and the previous cell state C_t–1. There are three learned gates: the forget f_t, input i_t, and output O_t. They are all produced by sigmoid layers (“σ” blocks). A tanh layer forms a candidate update \(\tilde{C}\)_t. Pointwise multiplications (“×”) and an addition (“+”) update the cell state on the top “conveyor” pathway. Arrows indicate vector flow; the legend marks pointwise operations, concatenation, and copies. This is the standard LSTM, emphasizing how gating enables selective memory retention and exposure. Used with permission granted by the author (Chris Olah), though it is slightly modified from its original form.

Trace the cell state’s top conveyor pathway as multiplication gates selectively preserve or modify information. section 4.2

How Sutskever's List covers it

How deep it goes

Explains the carry pathway, its contrast with hidden state, and its three gating controls without deriving the update equations.

Examples

Uses the conveyor-belt analogy, long-sequence tasks, and standard and original LSTM cell schematics.

What it assumes

Assumes familiarity with neural-network layers and sequence inputs while introducing LSTM memory pathways and gates.

What you can do after

Understand how LSTMs preserve, update, and expose information across time steps.

How it connects

Connects cell state with LSTM, hidden state, forget gate, input gate, output gate, residual connections, and gradient flow.

liveBookmanning.com

Read Sutskever's List on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Natural Language Processing in Action, Second EditionA cell state is the persistent recurrent representation maintained by an LSTM across sequence positions.
• 

  liveBookmanning.com
  Deep Learning with R, Third EditionA carry state is an information stream in an LSTM that transports a representation across sequence timesteps for later modification or use.
• 

  liveBookmanning.com
  Deep Learning with Python, Third EditionThe LSTM internal state that transports information across timesteps to preserve long-term context.
• 

  liveBookmanning.com
  Inside Deep Learning: Math, Algorithms, ModelsThe LSTM memory state that carries longer-term information separately from the hidden state.

Compare how 5 books explain carry state →

Related concepts

• LSTMpart of · An LSTM is a recurrent neural network architecture with gated memory mechanisms that control information flow and help retain or update information across sequences.

This explainer was generated by Manning's knowledge graph from the text of Sutskever's List. The section links point to the parts of the book it draws on.