A hidden state is an internal representation that carries context through a recurrent neural network. As the network reads a sequence one element at a time, it combines the current input with the previous state to produce an updated state and an output. The state acts as a form of memory: it lets later steps reflect earlier inputs rather than treating each element independently. section 3.4.3
Figure 3.14 A recurrent neural network unrolled across time. Each cell processes one element of the input sequence xt by combining the current input with the network’s previous internal state ht-1. This combination is performed using the same set of learned weights at every step: Un weights the current input, Wn transforms the previous internal state, and Vn maps the resulting state to the output. The internal state is passed forward from one step to the next, allowing information to accumulate across the sequence. Unrolling the network makes this temporal flow explicit and shows how a single computation is reused repeatedly over time.

The unrolled network shows how the state moves between steps while the same learned weights are reused. section 3.4.3
This memory has limits. Repeated transformations can weaken or amplify signals, making long-range context difficult to retain. The state also compresses past information into one evolving representation: later inputs blend with earlier signals, which can become entangled and hard to recover. section 3.4.3 section 3.5.2
Carrying a subject through a sentence
In “The book on the table near the window was written by a famous author,” a recurrent model must carry information about the subject across several intervening phrases. That information may become diffuse before it is needed. section 3.5.2
The term also describes contextualized token representations in attention-based models. There, attention combines information from tokens to produce hidden states that later layers refine. Unlike a recurrent state carried along a fixed path, these representations are recomputed at each position through token interactions. section 4.1.3
LSTMs and GRUs extend recurrent memory through gating mechanisms. Attention offers a different approach by allowing models to retrieve relevant information from positions across a sequence. section 3.6



