liveBook · Manning

Stemming

As explained in Elasticsearch in Action, Second Edition

A stemmer is a language-specific text-processing component that reduces related word forms to a root word.

Stemming improves full-text search when people use different forms of the same word. A stemmer reduces related forms to one root form, so “authors,” “authoring,” and “authored” can be indexed as “author.” This helps searches match language variations rather than relying only on exact terms. section 7.2.2 section 7.1.1

In Elasticsearch, stemming is part of text analysis. An analyzer tokenizes text and applies filters before terms enter the inverted index; stemming is also applied when a query runs against the same text field. Because languages differ, choose a language-appropriate stemmer or language analyzer for the documents being searched. section 4.5.1 section 7.3.7

Test a stemmer with the Analyze API

Use the _analyze API to inspect the tokens a stemmer produces. Here, the standard tokenizer splits the text and the stemmer filter changes “barking” to “bark”; the resulting tokens are “bark,” “is,” “my,” and “life.” section 7.8.1

Listing 7.49 Using a stemmer as the token filter
POST _analyze
{
  "tokenizer": "standard",
  "filter": ["stemmer"],
  "text": "barking is my life"
}

Run this request to see a stemmer token filter reduce “barking” to its root form. section 7.8.1

Configure stemming deliberately:

  • Use built-in language analyzers for supported languages; they can also use configurable stop-word handling. section 7.3.7
  • Exclude terms that must remain distinct. English stemming can reduce “authority” and “authorization” to “author,” preventing those original terms from reaching the inverted index. section 7.3.7
  • Test analyzer output before indexing, especially for domain vocabulary and multilingual content. section 4.5.1

Stemming is useful alongside tokenization, stop words, and synonyms, but it is not a substitute for them. It addresses word-form variation; synonyms and misspellings require separate analysis or search features. The goal is more relevant results across the varied wording users enter. section 7.2.2 section 7.1.1

How Elasticsearch in Action, Second Edition covers it

How deep it goes
Defines stemming, demonstrates analyzer output, and explains language configuration and stem exclusions without detailing stemming algorithms.
Examples
Demonstrates author word forms, barking analyzed through a stemmer filter, and authority and authorization protected by exclusions.
What it assumes
Introduces stemming from zero while assuming a need to configure and test Elasticsearch text analysis.
What you can do after
Apply it via Elasticsearch language analyzers or a stemmer token filter, then inspect results with the _analyze API.
How it connects
Connects stemming to analyzers, tokenization, token filters, stop words, synonyms, and the inverted index.

Also explained in

Compare how 4 books explain stemming →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Elasticsearch in Action, Second Edition. The section links point to the parts of the book it draws on.