An RNN’s feedback loop carries context from one time step to the next. At each step, the network combines the current sequence element with a hidden state summarizing earlier inputs. It updates that state and passes it forward—not the step’s output. Reusing the same computation lets an RNN process sequences without a fixed input length. section 4.1
Predicting the next character
An n-gram predicting what follows “hel” relies on counts within a fixed window. An RNN instead updates its hidden state as characters arrive, so its prediction can draw on context beyond that window. Character-by-character text generation made this difference vivid, though the hidden state is a learned summary, not a record of every earlier character. section 4.1
Figure 4.1 This image, from Chris Olah’s work (discussed in the next section), shows a chain-like structure representing an RNN. It illustrates how RNNs process data sequentially, with the output of one step feeding into the next. The recurrent, linked modules reflect the RNN’s ability to carry information forward, making it well suited to learning patterns that unfold over time, such as time series, text, or audio. Used with permission granted by the author (Chris Olah).
The linked modules show information carried through successive steps; the state passed forward provides the context for each new input. section 4.1
