Text augmentation creates altered or generated training examples to improve robustness or address limited data. It can help with overfitting, but language is unusually sensitive to small changes. section 14.5.3
Word deletion and swapping
Randomly deleting or swapping words can transform “The rain in Spain falls mainly on the plain” into “The rain Spain falls plain on the mainly.” Training on such edits may improve robustness to typos and grammar mistakes. section 14.5.3
Treat text augmentation cautiously because edits can change meaning:
- Word order is meaningful, so swapping two words can reverse a sentence’s meaning. section 14.5.3
- Synonym replacement can select the wrong sense of a word and produce brittle examples. section 14.5.3
- Seeking more naturally occurring text is usually preferable to investing heavily in augmentation techniques. section 14.5.3
- Generative models offer a newer approach by producing consistent and coherent text. section 14.5.3
