Carry state

As explained in Deep Learning with R, Third Edition

A carry state is an information stream in an LSTM that transports a representation across sequence timesteps for later modification or use.

A carry state is an information stream that transports a representation across sequence timesteps, where later operations can modify or use it. section 13.3.2

Within an LSTM cell, the carry state:

• Combines with the input and recurrent connection to influence the next output and next state. section 13.3.2
• Updates through three transformations associated with the input, forget, and candidate paths. section 13.3.2
• Uses the forget path to scale existing information and the input and candidate paths to add new information. section 13.3.2

Carry update

The update can be written as c_t+1 = i_t * k_t + c_t * f_t. The weights learn how these operations behave, so their intuitive roles are not fixed. section 13.3.2

Figure 13.8 Going from a SimpleRNN to an LSTM: adding a carry track

Compare the SimpleRNN cell with the added carry track to visualize how LSTM information flow expands across timesteps. section 13.3.2

How Deep Learning with R, Third Edition covers it

How deep it goes

Descends to the LSTM carry equation and transformations, while stopping short of assigning fixed meanings to learned operations.

Examples

Uses an LSTM anatomy figure and pseudocode showing output, gate, and carry-state calculations.

What it assumes

Builds from the SimpleRNN cell and recurrent state before adding the LSTM carry track.

What you can do after

Evaluate how an LSTM transports, forgets, and incorporates information through its carry pathway.

How it connects

Connects carry state with LSTM cells, SimpleRNN cells, recurrent connections, hidden state, and input and forget gates.

liveBookmanning.com

Read Deep Learning with R, Third Edition on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Sutskever's ListA cell state is the long-term memory pathway in an LSTM that carries information across time steps with selectively gated updates.
• 

  liveBookmanning.com
  Natural Language Processing in Action, Second EditionA cell state is the persistent recurrent representation maintained by an LSTM across sequence positions.
• 

  liveBookmanning.com
  Deep Learning with Python, Third EditionThe LSTM internal state that transports information across timesteps to preserve long-term context.
• 

  liveBookmanning.com
  Inside Deep Learning: Math, Algorithms, ModelsThe LSTM memory state that carries longer-term information separately from the hidden state.

Compare how 5 books explain carry state →

Related concepts

• LSTMpart of · An LSTM is a recurrent neural network architecture with gated memory mechanisms that control information flow and help retain or update information across sequences.

This explainer was generated by Manning's knowledge graph from the text of Deep Learning with R, Third Edition. The section links point to the parts of the book it draws on.