LSTM
A long short-term memory (LSTM) network is a recurrent neural-network architecture with gated memory cells that control information flow and model dependencies in sequential data.
Deep Learning with Python, Third Edition
A long short-term memory recurrent neural network architecture designed to capture long-range dependencies in sequences.
- How it is used
- Models ordered weather and text sequences for forecasting and classification.
- How deep it goes
- Explains carry-track updates and recurrent-layer output modes, then demonstrates regularization and stacking.
- Examples
- Compares Jena temperature forecasting and movie-review classification using one-hot, learned, and pretrained embeddings.
- What you can do after
- Apply it via Keras to train, stack, regularize, and evaluate sequence models.
Read how Deep Learning with Python, Third Edition explains LSTM →
Deep Learning with R, Third Edition
Long short-term memory is a gated recurrent-neural-network architecture designed to preserve and control information across sequence timesteps.
- How it is used
- Uses LSTMs for ordered-sequence forecasting and text classification, comparing them with simpler baselines.
- How deep it goes
- Explains recurrent input and output shapes, stacking, and the LSTM carry track, with architecture pseudocode.
- Examples
- Demonstrates Jena temperature forecasting and movie-review classification with a pretrained CBOW embedding.
- What you can do after
- Apply it via R Keras to build, train, and evaluate sequence models, including stacked and bidirectional variants.
Read how Deep Learning with R, Third Edition explains LSTM →
Inside Deep Learning: Math, Algorithms, Models
A gated recurrent neural-network architecture designed to learn long-range dependencies.
- How it is used
- Uses LSTMs to improve a recurrent name-language classifier and compare it with an ordinary RNN.
- How deep it goes
- Explains two states and three gates, then trains PyTorch’s built-in layer rather than implementing an LSTM cell from scratch.
- Examples
- Compares three-layer RNN and LSTM classifiers for names’ original languages, then proposes bidirectionality and LSTM–GRU timing experiments.
- What you can do after
- Apply it via PyTorch `nn.LSTM` in a sequence classifier and compare accuracy with an `nn.RNN` baseline.
Read how Inside Deep Learning: Math, Algorithms, Models explains LSTM →
Learn Generative AI with PyTorch
A recurrent neural network variant with long short-term memory units, mentioned here as an example of a deep learning model that cannot process raw text directly.
- How it is used
- Uses LSTMs for next-token prediction and text generation in a learned writing style.
- How deep it goes
- Explains cell state and training steps, while noting limits on extremely long-range dependencies.
- Examples
- Uses Anna Karenina as training text and pairs “how are you” with the shifted target “are you today.”
- What you can do after
- Apply it via PyTorch to train a text-generation model and generate tokens from a prompt.
Sutskever's List
Long short-term memory is a recurrent neural network architecture with gated state updates for modeling long-range sequence dependencies.
- How it is used
- Models ordered data when earlier context must influence later predictions despite many intervening steps.
- How deep it goes
- Explains the vanishing-gradient motivation and illustrates cell-state updates through gates, without deriving their equations.
- Examples
- Uses a France-to-French prediction and a single-cell diagram to show distant context and gated state flow.
- What you can do after
- Evaluate/understand it as a gated approach to long-range sequence context, distinct from regularization against overfitting.
Natural Language Processing in Action, Second Edition
Long short-term memory is a recurrent neural network architecture that uses gated state updates to preserve and control information over long sequences.
- How it is used
- Improves recurrent language modeling when important information must persist across long token sequences.
- How deep it goes
- Explains four gates, hidden and cell states, and update equations, while omitting full weight-matrix derivations and training details.
- Examples
- Uses LSTM gate equations and WikiText2 next-token prediction with <unk> masking as concrete implementation and evaluation examples.
- What you can do after
- Evaluate recurrent language-model behavior and interpret how gates control short-term and long-term sequence information.
Read how Natural Language Processing in Action, Second Edition explains LSTM →
LLMs in Production
A long short-term memory network is a recurrent neural network architecture that uses memory cells and gates to preserve or discard information across sequences.
- How it is used
- Uses recurrent memory and gates to classify variable-length language sequences such as tweets by sentiment.
- How deep it goes
- Explains memory cells, gating, bidirectionality, padding, and comparative training results, but does not derive gate equations.
- Examples
- Uses Twitter sentiment data, differently sized sentiment sentences, padded embeddings, and PyTorch RNN and LSTM implementations.
- What you can do after
- Build an LSTM sentiment classifier with preprocessing, embeddings, padding, and PyTorch model classes.
Machines that Think
A recurrent neural network architecture that uses gated memory cells to preserve and selectively update information across long sequences.
- How it is used
- Preserves relevant context across long sequences when dependencies extend beyond the immediate steps.
- How deep it goes
- Explains cell and hidden states plus forget, input, and output gates, without deriving their equations.
- Examples
- Uses a sentence linking “The book” to “was fascinating” across intervening words.
- What you can do after
- Understand how gating preserves sequence context and why sequential state updates still constrain access and parallelism.
Grokking Machine Learning
A long short-term memory network is a recurrent neural network architecture with gated memory cells designed to learn long-range dependencies in sequences.
- How it is used
- Processes text sequences for natural language processing tasks such as sentiment analysis, translation, and speech recognition.
- How deep it goes
- Introduces LSTM as an advanced RNN architecture but does not detail its internal memory-cell or gating mechanics.
- Examples
- Uses sentiment analysis, machine translation, speech recognition, text summarization, and topic-specific chatbots as applications.
- What you can do after
- Understand where LSTMs fit among sequence models and recognize suitable language-processing applications.
Also covered in
- Build an Advanced RAG Application (From Scratch)In early access (MEAP)
- Data Without Labels
- Transformers in Action
- Understanding AIIn early access (MEAP)
- Machine Learning Algorithms in Depth
- Grokking Machine Learning, Second EditionIn early access (MEAP)
What to know first
- language modelA language model is a computational model that represents patterns in linguistic sequences to assign probabilities, predict, generate, or evaluate language.
- accuracyAccuracy is the degree to which a result agrees with a specified truth, reference, or criterion, commonly measured as the proportion of evaluated predictions that are correct.
- natural language processingNatural language processing is the field of computing concerned with computational methods for analyzing, interpreting, transforming, and generating human language in text or speech.
- convolutional neural networkA convolutional neural network is a neural-network architecture that processes structured inputs with convolutional operations, typically using learned local filters to capture patterns.
Related concepts
- recurrent neural networkkind of · A recurrent neural network is a neural network that processes ordered inputs while carrying and updating an internal state across successive steps.
- vanishing gradient problemaddresses · The vanishing-gradient problem is a neural-network training difficulty in which gradients shrink during backpropagation until parameter updates in earlier layers or steps become ineffective.
- cell statehas part · The cell state is an LSTM’s persistent memory representation, carried across sequence steps and updated through gates to retain information over time.
- GRUalternative to · A gated recurrent unit (GRU) is a recurrent neural-network architecture that uses learned gates to regulate information flow and update a hidden state across sequence steps.
- maskused by · A mask is a representation or mechanism that selectively identifies, includes, excludes, reveals, or obscures specified parts of data, a system, or a computation.
- embeddingused by · An embedding is a numerical vector representation of an item in a feature space, encoding selected properties or relationships for computational tasks such as comparison or retrieval.
Questions about LSTM
What is LSTM?
A long short-term memory (LSTM) network is a recurrent neural-network architecture with gated memory cells that control information flow and model dependencies in sequential data.
Which Manning books explain LSTM?
Deep Learning with Python, Third Edition, Deep Learning with R, Third Edition, Inside Deep Learning: Math, Algorithms, Models, Learn Generative AI with PyTorch, Sutskever's List, Natural Language Processing in Action, Second Edition, LLMs in Production, Machines that Think and 1 more.
What should I know before learning LSTM?
Manning books that teach LSTM usually introduce language model, accuracy, natural language processing and convolutional neural network first.
What is the difference between LSTM and GRU?
A long short-term memory (LSTM) network is a recurrent neural-network architecture with gated memory cells that control information flow and model dependencies in sequential data. A gated recurrent unit (GRU) is a recurrent neural-network architecture that uses learned gates to regulate information flow and update a hidden state across sequence steps.
What is LSTM a kind of?
LSTM is a kind of recurrent neural network. A recurrent neural network is a neural network that processes ordered inputs while carrying and updating an internal state across successive steps.
What are the parts of LSTM?
LSTM includes cell state.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.








