liveBook · Manning

Mask

As explained in Deep Learning with R, Third Edition

Masking is the exclusion of designated sequence positions from downstream neural-network computations.

Masking excludes designated sequence positions from downstream neural-network computations. In padded text sequences, it prevents [PAD] tokens from influencing the recurrent representation used for prediction. section 14.5.3

Mask padded tokens in an LSTM

In Keras, set mask_zero = TRUE on the embedding layer when zero represents [PAD]. The mask is passed to the LSTM, which returns the last non-masked representation.\n\ninputs <- keras_input(shape = c(max_length), dtype = "int32")\noutputs <- inputs |>\n layer_embedding(input_dim = max_tokens, output_dim = 64L, mask_zero = TRUE) |>\n layer_bidirectional(layer_lstm(units = 64L)) |>\n layer_dense(1, activation = "sigmoid") section 14.5.3

How Deep Learning with R, Third Edition covers it

How deep it goes
Explains implicit embedding-layer masking and its effect on LSTM output, while deferring explicit mask construction.
Examples
Uses padded movie-review sequences and an R/Keras Embedding–bidirectional LSTM classifier.
What it assumes
Assumes integer-token sequences with zero assigned to [PAD] and teaches the masking mechanism through the Embedding layer.
What you can do after
Apply it via Keras by reserving zero for padding and enabling mask_zero on an Embedding layer.
How it connects
Connects masking with sequence padding, embeddings, recurrent neural networks, and LSTM representations.
Cover of Deep Learning with R, Third Edition
liveBookmanning.com
Read Deep Learning with R, Third Edition on liveBook Get the book at manning.com

Also explained in

Compare how 7 books explain mask →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Deep Learning with R, Third Edition. The section links point to the parts of the book it draws on.