Recurrent dropout
Recurrent dropout is dropout applied to recurrent computations using a mask that remains consistent across sequence timesteps.
Deep Learning with Python, Third Edition
A dropout variant for recurrent networks that applies a temporally consistent mask to recurrent computations.
- How it is used
- Regularizes recurrent networks, especially sequence models that would otherwise overfit.
- How deep it goes
- Explains temporal masks, recurrent activations, Keras arguments, convergence costs, and forecasting results without deriving the underlying mathematics.
- Examples
- Uses a dropout-regularized LSTM for Jena temperature forecasting with validation and test MAE comparisons.
- What you can do after
- Apply it via Keras recurrent layers by setting dropout and recurrent_dropout and evaluating overfitting on validation data.
Read how Deep Learning with Python, Third Edition explains recurrent dropout →
Deep Learning with R, Third Edition
A time-constant dropout mask is a dropout mask whose pattern remains fixed across the timesteps of a sequence.
- How it is used
- Regularizes recurrent layers to fight overfitting in sequence models.
- How deep it goes
- Explains mask timing, error propagation, Keras arguments, and a forecasting result without deriving the mechanism mathematically.
- Examples
- Uses a Jena temperature-forecasting LSTM with recurrent_dropout, layer dropout, validation metrics, and training curves.
- What you can do after
- Apply it via Keras by setting dropout and recurrent_dropout on recurrent layers and evaluating validation and test error.
Read how Deep Learning with R, Third Edition explains recurrent dropout →
Related concepts
- overfittingaddresses · Overfitting is a modeling condition in which a model fits patterns or idiosyncrasies in its training data too closely and consequently generalizes poorly to new data.
- dropoutkind of · Dropout is a neural-network regularization technique that randomly suppresses units, connections, or activations during training to reduce overfitting.
- dropout maskhas part · A dropout mask is a pattern specifying which values, units, or contributions are randomly retained or suppressed during training.
- Kerasimplemented by · Keras is an open-source, high-level deep-learning API for building, training, evaluating, and deploying neural-network models.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.