A gated recurrent unit (GRU) is a recurrent layer for ordered data. Like other recurrent neural networks, it processes a sequence one timestep at a time while carrying a state forward. It is a simpler, cheaper alternative to an LSTM, making it a practical choice for timeseries and text. section 20.1.6 section 15.1.1
Forecast temperature with stacked GRUs
With prepared sequence datasets and their input dimensions, build the model in R/Keras: inputs <- keras_input(shape = c(sequence_length, ncol_input_data)) outputs <- inputs |> layer_gru(32, recurrent_dropout = 0.5, return_sequences = TRUE) |> layer_gru(32, recurrent_dropout = 0.5) |> layer_dropout(0.5) |> layer_dense(1) model <- keras_model(inputs, outputs) model |> compile(optimizer = "adam", loss = "mse", metrics = "mae") The first GRU returns an output at every timestep so the second GRU receives a sequence. The second returns its last output for the single-value forecast. Recurrent dropout and a separate dropout layer regularize the model. In the reported run, its test mean absolute error was 2.40 degrees, about 8.3% better than the baseline; the extra layer helped, but only modestly. section 13.3.5
Listing 13.24 Training and evaluating a stacked GRU model
inputs <- keras_input(shape = c(sequence_length, ncol_input_data))
outputs <- inputs |>
layer_gru(32, recurrent_dropout = 0.5, return_sequences = TRUE) |>
layer_gru(32, recurrent_dropout = 0.5) |>
layer_dropout(0.5) |>
layer_dense(1)
model <- keras_model(inputs, outputs)
callbacks <- list(
callback_model_checkpoint(
"jena_stacked_gru_dropout.keras", save_best_only = TRUE
)
)
model |> compile(optimizer = "adam", loss = "mse", metrics = "mae")history <- model |> fit(
train_dataset,
epochs = 50,
validation_data = val_dataset,
callbacks = callbacks
)plot(history)model <- load_model("jena_stacked_gru_dropout.keras")
sprintf("Test MAE: %.2f", evaluate(model, test_dataset)[["mae"]])
[1] "Test MAE: 2.40"
The full example includes training, saving the best checkpoint, reloading it, and evaluating test error. section 13.3.5
The same layer can predict the next character of text. A small Shakespeare language model places a GRU after a character embedding and sets return_sequences = TRUE so a softmax classifier can make one prediction per input character. Its single GRU keeps the model simpler to train than a stack. section 15.1.1
Start with a commonsense baseline and a simpler model before paying for more recurrent layers. Stacking increases representational power and cost; it is not automatically worthwhile on a smaller problem. section 13.5
