liveBook · Manning

Recurrent neural network

As explained in Understanding AI

A recurrent neural network is a neural network that processes sequential inputs step by step while maintaining a state that carries information across the sequence.

A recurrent neural network (RNN) reads a sequence one step at a time while carrying an internal memory, or hidden state, from one step to the next. This lets it use earlier words when predicting what comes next. section 1.2

Predicting the weather

For “I left my umbrella at home, and now it’s…,” the RNN reads each part, updates its memory, and can use the accumulated context to predict “raining.” During training, repeated next-word prediction helps it learn grammar, common phrases, and relationships between words from data rather than hand-coded rules. section 1.2

Figure 1.2 Step-by-step processing of the sentence “I left my umbrella at home, and now it's…” by an RNN. At each step, the model reads one word (or group of words), updates its internal memory, and passes that memory forward to the next step. By step 3, the accumulated context that someone left an umbrella at home is sufficient for the model to predict a contextually appropriate continuation, such as "raining."

Follow the sentence word by word to see how the RNN updates its memory and carries context forward toward a prediction. section 1.2

The approach works well for nearby context but becomes fragile across long sequences: earlier details can fade as information passes through many intermediate steps. Attention addresses this bottleneck by focusing directly on relevant earlier words, leading toward transformer architectures. section 1.2.2 section 1.3

How Understanding AI covers it

How deep it goes
Explains sequential processing, hidden state, embeddings, and long-context limits, but defers mathematical or implementation detail.
Examples
Uses the umbrella sentence to illustrate context accumulation and contextually appropriate continuation.
What you can do after
Evaluate why sequential models can predict coherently yet lose important information across longer inputs.
How it connects
Connects hidden state and embeddings to next-word prediction, long-range dependencies, attention mechanisms, and transformer architectures.

Also explained in

Compare how 18 books explain recurrent neural network →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Understanding AI. The section links point to the parts of the book it draws on.