A recurrent neural network (RNN) processes ordered data one step at a time, carrying context forward rather than treating each input as independent. This makes it suited to sequences such as language, music, and sensor readings. section 3.4.3
At each step, the network combines the current input with a hidden state shaped by earlier inputs, then updates that state and produces an output. The same learned weights are reused at every step, allowing patterns to be processed across a sequence without separate parameters for each position. section 3.4.3
Figure 3.14 A recurrent neural network unrolled across time. Each cell processes one element of the input sequence xt by combining the current input with the network’s previous internal state ht-1. This combination is performed using the same set of learned weights at every step: Un weights the current input, Wn transforms the previous internal state, and Vn maps the resulting state to the output. The internal state is passed forward from one step to the next, allowing information to accumulate across the sequence. Unrolling the network makes this temporal flow explicit and shows how a single computation is reused repeatedly over time.

The unrolled diagram makes the repeated computation visible: each step combines a new input with the previous hidden state, using the same weights. section 3.4.3
This carried state supports short-range context, including local syntax and next-word prediction. But it compresses earlier information into one evolving representation. Signals may fade or grow unstable, and earlier details can become entangled with later context, making long-range dependencies difficult to retain or reinterpret. section 3.4.3 section 3.5.2
A subject carried across a sentence
In “The book on the table near the window was written by a famous author,” the subject must remain available across intervening phrases. An RNN carries it forward in its hidden state, where it may become harder to isolate. section 3.5.2
Recurrent networks established a way to model ordered information, but language also requires relationships to be revised as new words arrive. Even gated variants such as LSTMs and GRUs can struggle with this limitation, motivating architectures that move beyond strictly sequential processing. section 3.4.7 section 3.5.1

















