liveBook · Manning

Carry state

As explained in Deep Learning with Python, Third Edition

The LSTM internal state that transports information across timesteps to preserve long-term context.

Carry state is the LSTM track that transports information across timesteps, helping preserve long-term context. It extends the simpler recurrent state with a dedicated carry path.

Updating the carry

At each timestep, the LSTM combines a candidate update with the previous carry. An input gate controls what to write, while a forget gate controls what to retain: c_t+1 = i_t * k_t + c_t * f_t.

How Deep Learning with Python, Third Edition covers it

How deep it goes
Explains the carry update through input and forget gates, but stops short of implementation details.
Examples
Shows the carry track diagram and the gated pseudocode equation for updating c_t+1.
What it assumes
Assumes familiarity with recurrent states and SimpleRNNs before adding the LSTM carry track.
What you can do after
Evaluate/understand how LSTM gates retain, discard, and write information through the carry state.
How it connects
Connects LSTM carry state with recurrent state, SimpleRNN, gates, and vanishing gradients.
Cover of Deep Learning with Python, Third Edition
liveBookmanning.com
Read Deep Learning with Python, Third Edition on liveBook Get the book at manning.com

Also explained in

Compare how 5 books explain carry state →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Deep Learning with Python, Third Edition. The section links point to the parts of the book it draws on.