Attention assigns stronger connections to important word patterns, allowing a transformer to relate concepts separated by long stretches of text. It replaces recurrent processing with a connection matrix applied across the token sequence. section 9.1.1 section 9.2
Instead of moving through text one step at a time like an RNN or scanning local regions like a CNN, attention multiplies the whole sequence of token embeddings by an attention matrix. This removes recurrence and avoids recurrent vanishing- and exploding-gradient problems, but it requires fixed-length input and output sequences. section 9.1.1
Attention works within a transformer’s broader sequence representation:
- Byte pair encoding reduces a large vocabulary into reusable subword tokens. section 9.1.1
- Positional encoding preserves information about where each token occurs after recurrence is removed. section 9.1.1 section 9.2
- Consistent encoding shapes let transformer layers accept one another’s outputs and stack into deeper networks. section 9.1.2
- Inputs are truncated or padded to the required sequence length, limiting how much text each layer can process. section 9.1.2 section 9.1.1
Figure 9.3 The encoder attention matrix as connections between words
Visualizes the encoder attention matrix as connections between words, making long-range token relationships concrete. section 9.1.2
