A mask selectively blocks sequence positions during computation. In decoder self-attention, it prevents each position from accessing future tokens, preserving the previously known context required for sequential prediction. section 2.1.2
The mechanism serves the decoder in three connected stages:
- Place the mask in the decoder’s first sublayer, the masked multi-head self-attention layer. section 2.1.2
- Restrict attention so a prediction at one position depends only on previously known elements. section 2.1.2
- Support autoregressive tasks such as machine translation and text generation while the decoder produces one token at a time. section 2.1.2
Figure 2.6 The structure of the decoder in the transformer
Use the decoder architecture to locate masked self-attention before cross-attention and the feed-forward network. section 2.1.2
