Stop-word removal eliminates very common words with relatively little task-relevant meaning, reducing text dimensions and solution complexity. Use a customized list or a standard library, but inspect data quality and task context before filtering. section 7.6
Apply the technique selectively:
- Remove words such as “is,” “an,” “the,” “a,” “be,” “has,” “had,” and “it” when contextual information is not central. section 7.6
- Choose between a custom stop-word list and a predefined library, depending on the cleaning requirements. section 7.6
- Avoid automatic removal when short function words change the meaning of an answer or prediction. section 7.6
When context changes the decision
For the question “is it raining?”, the answer “it is” is complete; removing “it” and “is” would discard important contextual information. section 7.6
