To translate an English sentence into Spanish, the model must keep track of the English input while producing a new sentence one token at a time. An encoder turns the fixed source sequence into a representation; a decoder uses that representation and the Spanish tokens produced so far to predict the next token. The same pattern applies to question answering. section 15.2 section 15.6
English-to-Spanish translation
In a Keras RNN implementation, an embedding and bidirectional GRU encode the English sentence. A separate GRU starts from that encoded state, and a Dense softmax layer predicts probabilities over the Spanish vocabulary at each output step. The encoder can read in both directions because the whole source sentence is available; the decoder must not read future target tokens when predicting the next one. section 15.2.2
Keep training and generation distinct:
- During training, supply the source and the target sequence so far; train the decoder to predict the target sequence shifted one token forward. section 15.2
- During inference, begin with [start], feed each predicted token back into the decoder, and stop at [end] or the maximum sentence length. The target sentence is not available to guide predictions. section 15.2 section 15.2.2
Listing 15.13 Generating translations with a seq2seq RNN
import numpy as np spa_vocab = spanish_tokenizer.get_vocabulary() spa_index_lookup = dict(zip(range(len(spa_vocab)), spa_vocab)) def generate_translation(input_sentence): tokenized_input_sentence = english_tokenizer([input_sentence]) decoded_sentence = "[start]" for i in range(sequence_length): tokenized_target_sentence = spanish_tokenizer([decoded_sentence]) inputs = [tokenized_input_sentence, tokenized_target_sentence] next_token_predictions = seq2seq_rnn.predict(inputs, verbose=0) sampled_token_index = np.argmax(next_token_predictions[0, i, :]) sampled_token = spa_index_lookup[sampled_token_index] decoded_sentence += " " + sampled_token if sampled_token == "[end]": break return decoded_sentence test_eng_texts = [pair[0] for pair in test_pairs] for _ in range(5): input_sentence = random.choice(test_eng_texts) print("-") print(input_sentence) print(generate_translation(input_sentence))
The translation function makes the inference loop concrete: tokenize the English input, repeatedly predict a Spanish token, append it, and check for [end]. section 15.2.2
A Transformer can use the same two-part setup without an RNN: a TransformerEncoder produces a context-aware, order-aware source sequence, and a TransformerDecoder uses it alongside the target sequence to predict what comes next. Use an encoder alone when the task processes just one sequence rather than generating a target sequence. section 20.1.6
Do not treat next-token validation accuracy as translation quality. It measures predictions made with correct earlier target tokens supplied, whereas generation depends on the model’s own earlier predictions. For real translation evaluation, compare generated text with reference translations using a suitable metric such as BLEU. section 15.2.2
