A target sequence is the ordered sequence the decoder must produce, such as a translated sentence. During training, the decoder uses previous target tokens and the encoded source sequence to predict the next token. section 15.2
Translation from training to inference
For a translation task, the decoder receives the target sequence shifted by one position and learns to predict its next token. At inference, it starts with a seed such as "[start]", generates one token, feeds the partial sequence back, and stops at "[end]". section 15.2
When attention processes the whole target sequence, each target position produces a vector that queries every source vector. The resulting score matrix has one row per target position; softmax weights combine the relevant source vectors. section 15.3.1
Use these distinctions when working with target sequences:
- Treat training and inference differently: training has the target sequence, while inference builds it one token at a time. section 15.2
Figure 15.1 Sequence-to-sequence learning: the source sequence is processed by the encoder and is then sent to the decoder. The decoder looks at the target sequence so far and predicts the target sequence offset by one step in the future. During inference, we generate one target token at a time and feed it back into the decoder.
Trace how the source sequence reaches the decoder and how the target sequence is shifted during prediction. section 15.2
