Carry state is the persistent cell-state representation an LSTM passes from one sequence position to the next. It stores long-term patterns, while the hidden state emphasizes recently processed tokens and produces the current output. section 8.4.3
An LSTM updates carry state through learned gates:
- The forget gate determines which existing state elements remain available. section 8.4.3
- The input gate controls which candidate information enters the state. section 8.4.3
Reading a sequence
As tokens arrive, the carry state preserves a representation of meaning from earlier text and reuses it when processing later tokens. The hidden state remains the recurrent output needed at each time step. section 8.4.3
