liveBook · Manning

Mask

As explained in Deep Learning with Python, Third Edition

A technique for marking padded or irrelevant sequence positions so later layers ignore them.

A mask marks padded or irrelevant sequence positions so later layers ignore them. In a padded review sequence, masking prevents repeated [PAD] tokens from replacing the meaningful final representation, such as “awful.” section 14.5.3

Use masking in two practical ways:

  • Enable implicit masking with mask_zero=True in a Keras Embedding layer when token ID zero represents [PAD]. section 14.5.3
  • Pass an explicit mask to an LSTM when masking rules are custom or cannot rely on zero-valued token IDs. section 14.5.3

How Deep Learning with Python, Third Edition covers it

How deep it goes
Explains implicit zero-based masking and optional explicit masks, without detailing custom mask construction.
Examples
Demonstrates a padded movie review ending in “awful” and a Keras Embedding-to-Bidirectional-LSTM model.
What it assumes
Assumes padded token sequences and recurrent layers; teaches zero-valued padding masks through the embedding layer.
What you can do after
Operate/configure Keras sequence models that ignore zero-valued padding or accept explicit masks.
How it connects
Connects padding tokens with embedding layers, LSTM layers, and sequence classification outputs.
Cover of Deep Learning with Python, Third Edition
liveBookmanning.com
Read Deep Learning with Python, Third Edition on liveBook Get the book at manning.com

Also explained in

Compare how 7 books explain mask →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Deep Learning with Python, Third Edition. The section links point to the parts of the book it draws on.