An end-of-text token, written as EOS, marks the end of a tokenized sequence. During autoregressive decoding, the decoder generates tokens until it predicts EOS, then stops producing the sentence. section 10.5 section 9.1.2
A translation pipeline uses EOS in three connected steps:
- Adds BOS at the beginning and EOS at the end of every training phrase, so the model learns the sequence boundary. section 10.1.1
- Maps the resulting tokens to integer indexes through a vocabulary dictionary, alongside PAD and UNK entries. section 10.1.1
- Starts decoding with BOS, feeds earlier predictions back into the decoder, and terminates when the next prediction is EOS. section 9.1.2
English-to-French translation
For “I don't speak French.”, tokenization produces subword pieces such as “fr” and “ench</w>”. Adding BOS and EOS makes the boundary explicit before integer conversion. A decoder can then generate “Je ne parle pas français.” one token at a time and stop when it predicts EOS. section 10.1.1 section 10.5
Figure 9.4 The Transformer architecture. The encoder in the Transformer (left side of the diagram), which consists of N identical encoder layers, learns the meaning of the input sequence and converts it into vectors that represent its meaning. It then passes these vectors to the decoder (right side of the diagram), which consists of N identical decoder layers. The decoder constructs the output (e.g., the French translation of an English phrase) by predicting one token at a time, based on previous tokens in the sequence and vector representations from the encoder. The generator on the top right is the head attached to the output from the decoder so that the output is the probability distribution over all tokens in the target language (e.g., the French vocabulary).
Trace EOS through the Transformer’s decoder, where the generator predicts a probability distribution over target-language tokens. section 9.1.2
