SimpleRNN layer

Q: How deeply does Deep Learning with Python, Third Edition cover SimpleRNN layer?

Defines batch shapes, output modes, stacking, and vanishing gradients, but stops short of recommending SimpleRNN for most practical work.

As explained in Deep Learning with Python, Third Edition

A Keras recurrent layer that processes sequences by iterating a simple recurrent cell over timesteps.

SimpleRNN processes a batch of sequences one timestep at a time while maintaining a recurrent state. It accepts inputs shaped (batch_size, timesteps, input_features), or shape=(None, features) when sequence lengths may vary. section 13.3.2

Choose the output shape

For 120-step sequences with 14 features, SimpleRNN(16, return_sequences=False) produces (None, 16), keeping only the final output. Setting return_sequences=True produces (None, 120, 16), retaining one output per timestep. section 13.3.2

Use these practical rules when composing recurrent models:

Return only the last output when the layer is the final recurrent layer and the downstream task needs a sequence summary. section 13.3.2
Set return_sequences=True on every recurrent layer before another recurrent layer so the next layer receives a complete sequence. section 13.3.2
Treat SimpleRNN as a simple baseline: long-term dependencies are difficult to learn because of vanishing gradients, so GRU or LSTM is usually preferred. section 13.3.2 section 20.1.6

Listing 13.17 An RNN layer that returns only its last output step

>>> num_features = 14
>>> steps = 120
>>> inputs = keras.Input(shape=(steps, num_features))
>>> outputs = layers.SimpleRNN(16, return_sequences=False)(inputs)  #1
>>> print(outputs.shape)
(None, 16)

Inspect a minimal layer that returns only the final timestep output and prints its resulting shape. section 13.3.2

Listing 13.18 An RNN layer that returns its full output sequence

>>> num_features = 14
>>> steps = 120
>>> inputs = keras.Input(shape=(steps, num_features))
>>> outputs = layers.SimpleRNN(16, return_sequences=True)(inputs)  #1
>>> print(outputs.shape)
(None, 120, 16)

Compare the final-output configuration with a full sequence output that preserves all 120 timestep results. section 13.3.2

Listing 13.19 Stacking RNN layers

inputs = keras.Input(shape=(steps, num_features))
x = layers.SimpleRNN(16, return_sequences=True)(inputs)
x = layers.SimpleRNN(16, return_sequences=True)(x)
outputs = layers.SimpleRNN(16)(x)

Trace how return_sequences enables multiple SimpleRNN layers to be stacked. section 13.3.2

How Deep Learning with Python, Third Edition covers it

Cover of Deep Learning with Python, Third Edition

liveBook manning.com

Read Deep Learning with Python, Third Edition on liveBook Get the book at manning.com

Also explained in

liveBook manning.com

Deep Learning with R, Third EditionA SimpleRNN layer is a Keras recurrent layer that computes sequence representations using basic recurrent state updates.

Compare how 2 books explain SimpleRNN layer →

Related concepts

recurrent neural networkkind of · A recurrent neural network is a neural network that processes sequences by updating an internal state across successive inputs, allowing earlier inputs to influence later computations.
Kerasuses · Keras is an open-source, high-level deep-learning API for building, training, evaluating, and deploying neural-network models.
return_sequencesuses · return_sequences is a recurrent-layer setting that determines whether outputs are returned for every timestep or only the final timestep.
arrayuses · An array is an indexed, ordered collection of elements arranged in one or more dimensions, with elements accessed by position.
tensoruses · A tensor is a multidimensional, array-like structure of values indexed along zero or more dimensions and used for numerical computation.

This explainer was generated by Manning's knowledge graph from the text of Deep Learning with Python, Third Edition. The section links point to the parts of the book it draws on.