Stemming normalizes text by removing inflections or suffixes, reducing related words to truncated, root-like forms. In customer-review searches, this can improve keyword matching across word variations. section 5.5
Customer-review keyword search
Clean the review text, then stem each word before applying a positive-keyword function. A practical pipeline lowercases text, removes punctuation and extra whitespace, stems the words, and stores the result in cleaned_review_text. Compare the resulting matches with raw reviews because effectiveness varies by dataset and use case. section 5.5.5
# Using NLTK for stemming
from nltk.stem import RSLPStemmer
nltk.download('rslp')
stemmer = RSLPStemmer()
def stem_words(text):
words = text.split()
stemmed_words = [stemmer.stem(word) for word in words]
return ' '.join(stemmed_words)
df['cleaned_review_text'] = df['cleaned_review_text'].apply(stem_words)
Inspect the NLTK RSLPStemmer workflow for stemming Portuguese review text and assigning the transformed values to a dataframe column. section 5.5.5
Use stemming as one candidate step in a broader cleaning workflow:
- Remove punctuation and normalize whitespace before splitting reviews into words. section 5.5.5
- Choose stemming or lemmatization according to the search task; stemming is simpler, while lemmatization uses linguistic knowledge to produce base forms. section 5.5
- Do not assume every suggested cleaning step is necessary; test the cleaned results against the actual reviews. section 5.5.5
