Stemming
Stemming is a text-normalization technique that heuristically reduces inflected or derived word forms to a shared stem, which may not be a valid word.
Natural Language Processing in Action, Second Edition
Stemming is the reduction of inflected or derived word forms to a shared stem, often by removing suffixes without requiring the result to be a valid word.
- How it is used
- Groups related word forms during vocabulary normalization so indexed documents and search queries can match more broadly.
- How deep it goes
- Defines stems and demonstrates a suffix rule and NLTK implementations, without treating the rule as a complete solution.
- Examples
- Uses house and houses, a Python suffix rule, and NLTK Porter and Snowball outputs for a short phrase.
- What you can do after
- Apply it via NLTK stemmers and evaluate whether broader search recall justifies lost precision.
Read how Natural Language Processing in Action, Second Edition explains stemming →
Starting Data Analytics with Generative AI and Python
A text-normalization technique that reduces words to truncated root-like forms by applying linguistic rules.
- How it is used
- Improves keyword matching in unstructured text, including customer reviews used for sentiment-oriented searches.
- How deep it goes
- Defines root-like reduction and demonstrates a preprocessing function, but does not explain stemming algorithms internally.
- Examples
- Uses Portuguese customer reviews, NLTK’s RSLPStemmer, and a cleaned_review_text dataframe column.
- What you can do after
- Apply it via NLTK to preprocess review text before keyword searches, then inspect whether matching accuracy improves.
Read how Starting Data Analytics with Generative AI and Python explains stemming →
Elasticsearch in Action, Second Edition
A stemmer is a language-specific text-processing component that reduces related word forms to a root word.
- How it is used
- Improves full-text matching when users search with alternate grammatical forms of indexed words.
- How deep it goes
- Defines stemming, demonstrates analyzer output, and explains language configuration and stem exclusions without detailing stemming algorithms.
- Examples
- Demonstrates author word forms, barking analyzed through a stemmer filter, and authority and authorization protected by exclusions.
- What you can do after
- Apply it via Elasticsearch language analyzers or a stemmer token filter, then inspect results with the _analyze API.
Read how Elasticsearch in Action, Second Edition explains stemming →
Data Without Labels
Stemming is the heuristic reduction of inflected or derived words to a simplified stem that may not be a valid word.
- How it is used
- Normalizes related word forms during text cleaning before representation and machine-learning processing.
- How deep it goes
- Explains suffix-removal rules and Python execution, but stops short of detailed stemming algorithms or linguistic theory.
- Examples
- Uses NLTK's PorterStemmer with “eats,” “eating,” “studies,” and “study,” contrasting “studi” with lemmatized “study.”
- What you can do after
- Apply it via NLTK to tokenize words and generate simplified stems for text preprocessing.
Related concepts
- lemmatizationalternative to · Lemmatization is the linguistic normalization of inflected word forms to their canonical dictionary forms, called lemmas.
- normalizationused by · Normalization is the process of transforming a value, representation, or data structure to satisfy a specified standard, scale, or structural constraint.
- text analyzerused by · A text analyzer is a component that transforms text into terms or other linguistic representations for indexing and searching.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.