A recurrent dropout mask selectively drops recurrent values while keeping the same pattern at every timestep. This time-constant mask regularizes recurrent networks without disrupting error propagation through time; a mask that changes each timestep can hinder learning. section 13.3.4
Use recurrent dropout by:
- Setting a recurrent layer’s recurrent_dropout rate for recurrent units. section 13.3.4
- Keeping input-unit dropout separate from recurrent dropout; Keras exposes them as dropout and recurrent_dropout. section 13.3.4
- Expecting longer convergence and using a larger model when dropout reduces overfitting. section 13.3.4
Listing 13.23 Training and evaluating a dropout-regularized LSTM
inputs <- keras_input(shape = c(sequence_length, ncol_input_data))
outputs <- inputs |>
layer_lstm(32, recurrent_dropout = 0.25) |>
layer_dropout(0.5) |> #1
layer_dense(1)
model <- keras_model(inputs, outputs)
callbacks = list(
callback_model_checkpoint("jena_lstm_dropout.keras", save_best_only = TRUE)
)
compile(model, optimizer = "adam", loss = "mse", metrics = "mae")history <- model |> fit(
train_dataset,
epochs = 50,
validation_data = val_dataset,
callbacks = callbacks
)
A runnable R/Keras LSTM example combines recurrent dropout with ordinary dropout, checkpointing, and extended training on temperature forecasting. section 13.3.4
