A recurrent neural network combines each current input with an evolving hidden state: a fixed-size internal summary of earlier sequence elements. Later predictions can use this stored context, not just the current input. section 12.1.1
At each update, the network must retain, modify, or discard information. Long-distance relationships can weaken when relevant context fails to survive successive updates; for example, the subject “book” must remain connected to “was” across intervening clauses. State updates also depend on preceding states, so the sequence is processed in order. LSTMs and GRUs preserve information more effectively, but retain recurrent state propagation. Transformers instead use self-attention to model token relationships directly, with positional encodings preserving order. section 12.1.1 section 12.3.2



