LSTM
An LSTM is a recurrent neural network architecture with gated memory mechanisms that control information flow and help retain or update information across sequences.
Deep Learning with Python, Third Edition
A long short-term memory recurrent neural network architecture designed to capture long-range dependencies in sequences.
- How it is used
- Models ordered sequences for temperature forecasting and token-sequence classification.
- How deep it goes
- Works through Keras models, output shapes, masking, and regularization, with gate-update pseudocode rather than a from-scratch implementation.
- Examples
- Uses Jena temperature forecasting and movie-review classification to compare model results, input encodings, and training costs.
- What you can do after
- Apply it via Keras LSTM layers to build and evaluate sequence predictors, including stacked or bidirectional models.
Read how Deep Learning with Python, Third Edition explains LSTM →
Deep Learning with R, Third Edition
Long short-term memory is a gated recurrent-neural-network architecture designed to preserve and control information across sequence timesteps.
- How it is used
- Models ordered sequences for timeseries forecasting and text classification when temporal or word-order information matters.
- How deep it goes
- Works through Keras models, output shapes, stacking, and a carry-track update, without a full mathematical derivation.
- Examples
- Uses Jena temperature forecasting, stacked recurrent-layer configurations, and movie-review classification with pretrained CBOW embeddings.
- What you can do after
- Apply it via R/Keras to build and evaluate sequence models, choosing last-output or full-sequence mode for the architecture.
Read how Deep Learning with R, Third Edition explains LSTM →
Inside Deep Learning: Math, Algorithms, Models
A gated recurrent neural-network architecture designed to learn long-range dependencies.
- How it is used
- Uses LSTMs as a practical default recurrent architecture when ordinary RNNs are difficult to train across long sequences.
- How deep it goes
- Explains hidden and context states, three gates, bounded activations, PyTorch construction, and training behavior without deriving full gate equations.
- Examples
- Demonstrates LSTM classification with embedded names, three layers, LastTimeStep reduction, accuracy curves, and an RNN comparison.
- What you can do after
- Apply it via PyTorch by replacing nn.RNN with nn.LSTM, preserving sequence reduction, clipping gradients, and evaluating accuracy.
Read how Inside Deep Learning: Math, Algorithms, Models explains LSTM →
Learn Generative AI with PyTorch
A recurrent neural network variant with long short-term memory units, mentioned here as an example of a deep learning model that cannot process raw text directly.
- How it is used
- Uses LSTMs to predict successive text tokens and generate prose that reflects training-data patterns.
- How deep it goes
- Explains cell-state memory and walks through data preparation, model layers, training, and generation without detailing cell gates.
- Examples
- Uses Anna Karenina, 100-token training sequences, a shifted-token example, and prompt-based generation.
- What you can do after
- Build it from scratch with PyTorch’s LSTM layer, then generate text by repeatedly predicting and appending tokens.
Sutskever's List
Long short-term memory is a recurrent neural network architecture with gated state updates for modeling long-range sequence dependencies.
- How it is used
- Models ordered data when earlier context must influence later predictions despite many intervening steps.
- How deep it goes
- Explains the vanishing-gradient motivation and illustrates cell-state updates through gates, without deriving their equations.
- Examples
- Uses a France-to-French prediction and a single-cell diagram to show distant context and gated state flow.
- What you can do after
- Evaluate/understand it as a gated approach to long-range sequence context, distinct from regularization against overfitting.
Natural Language Processing in Action, Second Edition
Long short-term memory is a recurrent neural network architecture that uses gated state updates to preserve and control information over long sequences.
- How it is used
- Improves recurrent language modeling when important information must persist across long token sequences.
- How deep it goes
- Explains four gates, hidden and cell states, and update equations, while omitting full weight-matrix derivations and training details.
- Examples
- Uses LSTM gate equations and WikiText2 next-token prediction with <unk> masking as concrete implementation and evaluation examples.
- What you can do after
- Evaluate recurrent language-model behavior and interpret how gates control short-term and long-term sequence information.
Read how Natural Language Processing in Action, Second Edition explains LSTM →
LLMs in Production
A long short-term memory network is a recurrent neural network architecture that uses memory cells and gates to preserve or discard information across sequences.
- How it is used
- Uses recurrent memory and gates to classify variable-length language sequences such as tweets by sentiment.
- How deep it goes
- Explains memory cells, gating, bidirectionality, padding, and comparative training results, but does not derive gate equations.
- Examples
- Uses Twitter sentiment data, differently sized sentiment sentences, padded embeddings, and PyTorch RNN and LSTM implementations.
- What you can do after
- Build an LSTM sentiment classifier with preprocessing, embeddings, padding, and PyTorch model classes.
Grokking Machine Learning
A long short-term memory network is a recurrent neural network architecture with gated memory cells designed to learn long-range dependencies in sequences.
- How it is used
- Processes text sequences for natural language processing tasks such as sentiment analysis, translation, and speech recognition.
- How deep it goes
- Introduces LSTM as an advanced RNN architecture but does not detail its internal memory-cell or gating mechanics.
- Examples
- Uses sentiment analysis, machine translation, speech recognition, text summarization, and topic-specific chatbots as applications.
- What you can do after
- Understand where LSTMs fit among sequence models and recognize suitable language-processing applications.
Related concepts
- recurrent neural networkkind of · A recurrent neural network is a neural network that processes sequences by updating an internal state across successive inputs, allowing earlier inputs to influence later computations.
- vanishing gradient problemaddresses · The vanishing-gradient problem is a neural-network training difficulty in which gradients shrink toward zero as they propagate backward, making updates to earlier layers or distant sequence steps ineffective.
- carry statehas part · A carry state is an LSTM’s persistent long-term memory representation, transported across sequence positions through selectively gated updates.
- GRUalternative to · A gated recurrent unit (GRU) is a recurrent neural-network architecture that uses learned gates to regulate information flow and update a hidden state across sequence steps.
- maskused by · Masking is the selective representation, concealment, inclusion, exclusion, or restriction of parts of data or a system according to specified criteria.
- embeddingused by · A numerical vector representation of an item or input that encodes features or relationships useful for computational tasks.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.