RNNsearch is an attention-based recurrent encoder-decoder for neural machine translation. It addresses the lossy compression of long sentences by replacing one fixed “thought vector” with source annotations that the decoder consults dynamically. section 5.3
Its translation process combines:
- A bidirectional GRU encoder, whose joined forward and backward outputs represent every source word. section 5.3
- Bahdanau attention, which scores source-word relevance, turns scores into weights, and forms a weighted context vector. section 5.3
- A 1,000-unit GRU decoder, which combines its state, previous output, and current context to predict the next word. section 5.3
Aligning reordered phrases
For “European Economic Area” → “zone économique européenne,” attention first links “zone” to “Area,” then revisits earlier source words to complete the reordered phrase.
