Recurrent neural network
A recurrent neural network is a neural network that processes sequences by updating an internal state across successive inputs, allowing earlier inputs to influence later computations.
Natural Language Processing in Action, Second Edition
A recurrent neural network is a neural network architecture that processes sequences while maintaining a state influenced by earlier inputs.
- How it is used
- Processes token or numerical sequences for classification, tagging, next-step prediction, and text generation.
- How deep it goes
- Works from hidden-state intuition through a PyTorch model and training loop to backpropagation through time and gradient limitations.
- Examples
- Uses surname classification, prefix-by-prefix predictions, and a WikiText2 word-level language model.
- What you can do after
- Build it from scratch in PyTorch, train it on character sequences, and inspect predictions and hidden states.
Read how Natural Language Processing in Action, Second Edition explains recurrent neural network →
Sutskever's List
A recurrent neural network is a neural network architecture that processes sequences by maintaining a state updated across successive inputs.
- How it is used
- Models ordered text, speech, and time series by carrying context forward for later predictions.
- How deep it goes
- Explains hidden state, unrolled training, vanishing gradients, LSTM remedies, and translation bottlenecks without a formal derivation.
- Examples
- Uses a distant clue in a sentence, character-level essay generation, speech recognition, and machine translation.
- What you can do after
- Evaluate/understand when recurrent state helps sequence prediction and why long dependencies, overfitting, or fixed-length encoding can limit it.
Read how Sutskever's List explains recurrent neural network →
Deep Learning with Python, Third Edition
A neural network architecture that processes sequences by feeding information from one step to the next.
- How it is used
- Models ordered inputs for temperature forecasting, language modeling, and sequence-to-sequence translation.
- How deep it goes
- Works through a timestep-and-state loop in NumPy, then applies Keras recurrent layers to forecasting and text tasks.
- Examples
- Uses Jena weather readings, Shakespeare character prediction, and English-to-Spanish translation.
- What you can do after
- Build it from scratch as a NumPy loop and apply it via Keras recurrent layers to sequence tasks.
Read how Deep Learning with Python, Third Edition explains recurrent neural network →
Deep Learning with R, Third Edition
A recurrent neural network is a neural-network model that processes sequences recurrently by carrying information through an internal state across positions.
- How it is used
- Models ordered inputs for timeseries forecasting and token-by-token language prediction.
- How deep it goes
- Works through a zero-state forward loop, Keras output shapes, LSTM forecasting, and encoder–decoder translation without a formal derivation.
- Examples
- Uses Jena temperature forecasting, a hand-written R state-update loop, character-level language modeling, and English-to-Spanish translation.
- What you can do after
- Apply it via R/Keras recurrent layers to forecast an ordered series or produce predictions at each sequence position.
Read how Deep Learning with R, Third Edition explains recurrent neural network →
Inside Deep Learning: Math, Algorithms, Models
A neural network that processes sequential data by reusing parameters across time steps and carrying a hidden state forward.
- How it is used
- Processes ordered, variable-length data for sequence classification and other sequence predictions.
- How deep it goes
- Explains weight sharing, hidden-state recurrence, tensor shapes, batching, packing, and PyTorch assembly, but defers detailed gated-unit internals.
- Examples
- Uses weather vectors, character sequences, language-name classification, unrolled diagrams, and packed PyTorch training models.
- What you can do after
- Build it from scratch with PyTorch embeddings, nn.RNN, LastTimeStep, padding, packing, and a linear classifier.
Read how Inside Deep Learning: Math, Algorithms, Models explains recurrent neural network →
Graph Neural Networks in Action
A recurrent neural network is a neural network architecture that processes sequences using recurrent state updates.
- How it is used
- Provides a temporal baseline for predicting future body-joint sensor positions from past frames without edge data.
- How deep it goes
- Explains hidden-state updates and backpropagation through time, then implements an nn.RNN; leaves difficult gradient computations to deep-learning packages.
- Examples
- Uses body-joint sensor trajectories, an unfolded recurrent-state diagram, and PyTorch model, training, and forecasting code.
- What you can do after
- Apply it via PyTorch nn.RNN to forecast sensor positions and recognize when missing relational information limits predictions.
Read how Graph Neural Networks in Action explains recurrent neural network →
Learn Generative AI with PyTorch
A neural network architecture for sequential data that processes inputs with loops so information can persist across time steps.
- How it is used
- Uses recurrent models to introduce sequential prediction and generate text while preserving context from earlier inputs.
- How deep it goes
- Explains hidden-state updates through a time-step example and diagram, then identifies long-range and training-speed limitations without deriving the update equations.
- Examples
- Works through next-word prediction for “a frog has four legs” and uses a reversed football result to show why word order matters.
- What you can do after
- Build and train an LSTM-based text generator, then control its output with temperature and top-K sampling.
Read how Learn Generative AI with PyTorch explains recurrent neural network →
LLMs in Production
A recurrent neural network is a neural network that processes ordered inputs while maintaining a state that carries information across sequence positions.
- How it is used
- Processes ordered language inputs while preserving information across sequence positions for language modeling and sentiment analysis.
- How deep it goes
- Explains state, variable-length inputs, and gradient failure modes, then points to LSTM remedies without deriving recurrent internals.
- Examples
- Uses paired sentiment sentences and a tokenized Twitter dataset with sentiment labels to illustrate sequence modeling.
- What you can do after
- Evaluate sequence-model trade-offs, especially variable-length support and gradient instability, before choosing an RNN or LSTM.
Read how LLMs in Production explains recurrent neural network →
Grokking Machine Learning
A recurrent neural network is a neural network architecture that processes sequences by retaining and updating state across successive inputs.
- How it is used
- Processes text and other sequential inputs for language-understanding applications.
- How deep it goes
- Defines the architecture and its language-processing role but stops short of explaining its internal equations or training procedure.
- Examples
- Uses sentiment analysis, machine translation, speech recognition, text summarization, and chatbots as application examples.
- What you can do after
- Evaluate understanding of sequence-processing applications such as sentiment analysis, translation, and speech recognition.
Read how Grokking Machine Learning explains recurrent neural network →
Data Without Labels
A recurrent neural network is a neural-network architecture designed to model sequences through recurrent state or connections.
- How it is used
- Handles ordered datasets whose temporal dependencies influence pattern recognition.
- How deep it goes
- Defines recurrent state, shared weights, outputs, and gradient limits, but defers detailed LSTM and gated-unit treatment.
- Examples
- Uses time-series data, videos, natural language, speech recognition, and other ordered datasets as application contexts.
- What you can do after
- Evaluate sequence-modeling choices by relating order, recurrent memory, and long-term dependency limits to data requirements.
Read how Data Without Labels explains recurrent neural network →
Related concepts
- LSTMincludes · An LSTM is a recurrent neural network architecture with gated memory mechanisms that control information flow and help retain or update information across sequences.
- sequenceuses · A sequence is an ordered collection of elements whose positions determine their order of traversal or processing.
- language modelkind of · A language model is a computational model that represents patterns in linguistic tokens to estimate sequence probabilities and predict or generate language from context.
- hidden stateuses · A hidden state is an internal numerical representation in a neural model that encodes information processed up to a particular position, step, or layer.
- bidirectional rnnused by · A bidirectional recurrent neural network processes a sequence in forward and reverse directions and combines the resulting representations.
- statehas part · State is information or a configuration describing the condition of a system, entity, or computation at a particular time and potentially influencing subsequent behavior or decisions.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.