Scale invariance means recognizing a pattern despite changes in how spread out it is across a sequence. In text, convolutional neural networks can detect related word patterns at different scales, unlike fully connected networks and TF–IDF vectors. section 7.1
Patterns with different spacing
A text CNN can recognize a language pattern whether its words are tightly grouped or more spread out. This complements translation invariance, which handles the same pattern appearing in different positions. section 7.1
Scale invariance matters because it helps models:
- Generalize beyond patterns appearing at one fixed location or extent. section 7.1
- Support natural language processing tasks such as named entity recognition and word tagging. section 7.1
