Translation invariance means recognizing a pattern even when its position changes within an input. In text, the same word pattern can appear near the beginning, middle, or end of a sentence. section 7.1
A pattern moving through a sentence
A fully connected network may fail to generalize when a phrase moves from the beginning to the end of a sentence. A convolutional neural network can recognize the pattern wherever it occurs. section 7.1
Translation invariance matters because it:
- Lets convolutional neural networks detect patterns regardless of their location in text. section 7.1
- Supports natural language processing tasks such as named entity recognition and other word-tagging problems. section 7.1
- Differs from scale invariance, which concerns how spread out a pattern is. section 7.1
