A stop word is a frequent word such as “a,” “an,” “the,” “and,” or “is” that contributes little to distinguishing relevant documents. Text analysis can remove these words before indexing so searches focus on more informative terms. section 7.2.2 section 3.4.2
An analyzer applies stop-word handling alongside tokenization, stemming, and synonym processing. Language analyzers can use built-in stop-word lists or accept a list suited to an application’s search requirements. section 7.2.2 section 7.3.7
Use stop-word filtering when:
- Common function words add little retrieval discrimination and can be ignored during analysis. section 3.4.2
- A language analyzer needs a built-in or custom list of words excluded from the inverted index. section 7.3.7
- Search behavior requires application-specific choices rather than a language analyzer’s default stop-word set. section 7.3.7
Listing 7.19 Index with custom stop words on the english analyzer
PUT index_with_custom_english_analyzer #1
{
"settings": {
"analysis": {
"analyzer": {
"index_with_custom_english_analyzer":{ #2
"type":"english", #3
"stopwords":["a","an","is","and","for"] #4
}
}
}
}
}
Configure an English analyzer with the custom stop words “a,” “an,” “is,” “and,” and “for.” section 7.3.7
Test the custom analyzer
Analyze “A dog is for a life” with the configured analyzer. The resulting tokens are “dog” and “life”; the listed stop words are removed. section 7.3.7
Listing 7.20 Testing the custom stop words for the english analyzer
POST index_with_custom_english_analyzer/_analyze
{
"text":"A dog is for a life",
"analyzer":"index_with_custom_english_analyzer"
}
Run the _analyze request against the custom index to verify which tokens remain after stop-word filtering. section 7.3.7
Check the analyzer’s defaults before enabling filtering: the standard analyzer leaves its stopwords setting at _none_ by default. Removing words can change what reaches the inverted index, so choose the list according to search requirements. section 3.4.2 section 7.3.7
