Machine translation automatically converts text or speech from one human language to another. It is a long-standing language technology: early systems used programmed language rules, while later approaches learned translation patterns from text. section 1.10
For neural translation, an input sentence is split into tokens and represented mathematically. A model processes that representation, then decodes its output as text. Attention can help the model focus on relevant parts of the input while generating a translation. section 1.2
An English sentence translated into Spanish
The input “The cat wore red socks” can be encoded and processed by a model, which decodes the Spanish output “El gato usó calcetines rojos.” section 1.2
Figure 1.2 The distribution of attention for the word “it” in different contexts.
The figure illustrates how attention to the word “it” can differ across contexts. section 1.2
