How deeply does Deep Learning with R, Third Edition cover LSTM?

Works through Keras models, output shapes, stacking, and a carry-track update, without a full mathematical derivation.

LSTM

As explained in Deep Learning with R, Third Edition

Long short-term memory is a gated recurrent-neural-network architecture designed to preserve and control information across sequence timesteps.

An LSTM (long short-term memory) layer is a recurrent neural network layer for ordered data. It processes a sequence one timestep at a time while maintaining state, making it useful when earlier inputs matter to later predictions. In temperature forecasting, flattening the timeseries discarded its temporal structure; an LSTM preserved it and beat the commonsense baseline. section 13.3 section 20.1.6

Forecast temperature with an LSTM

In R/Keras, a compact model is `inputs <- keras_input(shape = c(sequence_length, ncol_input_data)); outputs <- inputs |> layer_lstm(16) |> layer_dense(1); model <- keras_model(inputs, outputs)`. Compile it with Adam, mean squared error loss, and mean absolute error (MAE) as a metric. Train against a validation dataset, save the best checkpoint, then evaluate it on the test dataset. The saved model produced a test MAE of 2.52 degrees. section 13.3

Two choices shape how a recurrent layer fits into a model:

Inputs have shape `(batch_size, timesteps, input_features)`. Use an unspecified timestep dimension for variable-length sequences; specify the length when it is fixed to make output shapes clearer and potentially improve performance. section 13.3.2
Return the last output for a single sequence-level prediction. Set `return_sequences = TRUE` on every LSTM that feeds another recurrent layer, so the next layer receives an output at each timestep. section 13.3.2 section 20.1.6

The simpler `SimpleRNN` struggles to learn long-term dependencies because of vanishing gradients. An LSTM adds a carry track whose next value combines new information with retained information: `c_t+1 = i_t * k_t + c_t * f_t`. Its learned components control those contributions rather than relying only on a repeatedly updated simple state. section 13.3.2

Listing 13.22 Pseudocode details of the LSTM architecture (2/2)

c_t+1 = i_t * k_t + c_t * f_t

The carry-track update shows how an LSTM combines a new contribution with information retained from the preceding timestep. section 13.3.2

LSTMs also accept sequences of word embeddings. For movie-review classification, a bidirectional LSTM with a pretrained CBOW embedding reached about 0.891 test accuracy. That result was slightly below a bigram model: learning from word order can demand more training data. For a cheaper recurrent alternative, consider GRU; for difficult long-text dependencies, Transformers offer another approach. Compare each against a simple baseline before paying for a larger model. section 14.5.5 section 20.1.6 section 14.1 section 13.5

How Deep Learning with R, Third Edition covers it

Cover of Deep Learning with R, Third Edition

liveBook manning.com

Read Deep Learning with R, Third Edition on liveBook Get the book at manning.com

Also explained in

liveBook manning.com

Deep Learning with Python, Third EditionA long short-term memory recurrent neural network architecture designed to capture long-range dependencies in sequences.
liveBook manning.com

Inside Deep Learning: Math, Algorithms, ModelsA gated recurrent neural-network architecture designed to learn long-range dependencies.
liveBook manning.com

Learn Generative AI with PyTorchA recurrent neural network variant with long short-term memory units, mentioned here as an example of a deep learning model that cannot process raw text directly.
liveBook manning.com

Sutskever's ListLong short-term memory is a recurrent neural network architecture with gated state updates for modeling long-range sequence dependencies.
liveBook manning.com

Natural Language Processing in Action, Second EditionLong short-term memory is a recurrent neural network architecture that uses gated state updates to preserve and control information over long sequences.
liveBook manning.com

LLMs in ProductionA long short-term memory network is a recurrent neural network architecture that uses memory cells and gates to preserve or discard information across sequences.
liveBook manning.com

Grokking Machine LearningA long short-term memory network is a recurrent neural network architecture with gated memory cells designed to learn long-range dependencies in sequences.

Compare how 8 books explain LSTM →

Related concepts

recurrent neural networkkind of · A recurrent neural network is a neural network that processes sequences by updating an internal state across successive inputs, allowing earlier inputs to influence later computations.
vanishing gradient problemaddresses · The vanishing-gradient problem is a neural-network training difficulty in which gradients shrink toward zero as they propagate backward, making updates to earlier layers or distant sequence steps ineffective.
carry statehas part · A carry state is an LSTM’s persistent long-term memory representation, transported across sequence positions through selectively gated updates.
embeddingused by · A numerical vector representation of an item or input that encodes features or relationships useful for computational tasks.
maskused by · Masking is the selective representation, concealment, inclusion, exclusion, or restriction of parts of data or a system according to specified criteria.
GRUalternative to · A gated recurrent unit (GRU) is a recurrent neural-network architecture that uses learned gates to regulate information flow and update a hidden state across sequence steps.

This explainer was generated by Manning's knowledge graph from the text of Deep Learning with R, Third Edition. The section links point to the parts of the book it draws on.