liveBook · Manning

LSTM

As explained in Machines that Think

A recurrent neural network architecture that uses gated memory cells to preserve and selectively update information across long sequences.

Long Short-Term Memory (LSTM) is a recurrent neural network architecture for sequences where important information may be separated by many steps. Compared with a basic recurrent neural network, whose signals can fade or grow uncontrollably as they pass through a sequence, an LSTM uses gates to regulate what information to keep, update, or forget. section 3.4.4

Figure 3.15 An LSTM neuron. At each time step, the cell receives the input xt, the previous hidden state ht-1, and the previous cell state Ct-1. The forget gate uses these signals to determine how much of the old memory to keep, while the input gate and the candidate update, propose and regulate the new information to add. Together, these components produce the updated cell state Ct, which carries long-term memory forward. The output gate then controls how much of this updated memory becomes visible by shaping the hidden state ht. This division between a protected memory pathway and an exposed hidden state allows LSTMs to maintain information across many time steps while still responding flexibly to new inputs.

An LSTM separates a cell state that carries memory from a hidden state that exposes information. Forget, input, and output gates control how old memory is retained, new information is added, and updated memory is revealed. section 3.4.4

Keeping a sentence’s subject in view

In “The book that I borrowed last week was fascinating,” interpreting “was fascinating” depends on retaining “the book” across the intervening words. An LSTM can preserve that information while filtering less relevant details. section 3.4.4

This longer reach supports sequence tasks such as machine translation and speech recognition. It does not remove the constraints of sequential processing: an LSTM updates one evolving state at a time, limiting parallelism, and distant information can become compressed or interfere with other dependencies. section 3.5.3

How Machines that Think covers it

How deep it goes
Explains cell and hidden states plus forget, input, and output gates, without deriving their equations.
Examples
Uses a sentence linking “The book” to “was fascinating” across intervening words.
What you can do after
Understand how gating preserves sequence context and why sequential state updates still constrain access and parallelism.
How it connects
Connects recurrent neural networks and GRUs with sequence-to-sequence models and attention.

Also explained in

Compare how 9 books explain LSTM →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Machines that Think. The section links point to the parts of the book it draws on.