liveBook · Manning

Custom analyzer

As explained in Elasticsearch in Action, Second Edition

An Elasticsearch analyzer assembled with user-selected character filters, tokenizer, and token filters.

A custom analyzer assembles user-selected character filters, one tokenizer, and zero or more token filters. Use it when a built-in analyzer cannot produce the tokens your search requirements need. Character filters clean or rewrite input first, the tokenizer splits it, and token filters transform the resulting tokens. section 7.4 section 7.9

Normalize HTML and uppercase words

Create the analyzer in index settings: PUT index_with_custom_analyzer { "settings": { "analysis": { "analyzer": { "custom_analyzer": { "type": "custom", "char_filter": ["html_strip"], "tokenizer": "standard", "filter": ["uppercase"] } } } } } Test it with `POST index_with_custom_analyzer/_analyze` and the text `<H1>HELLO, WoRLD</H1>`. HTML tags are removed, standard tokenization produces two words, and the uppercase filter returns `HELLO` and `WORLD`. section 7.4

Define named analysis components alongside the analyzer when defaults are insufficient. A mapping character filter can convert symbols such as α, β, and γ into words, while a keep filter can discard every token except `alpha`, `beta`, and `gamma`. Test the complete pipeline before relying on its indexed output. section 7.4.1

For quick experiments, the `_analyze` API can mix a tokenizer and filters without creating a named index analyzer. A path hierarchy tokenizer with an uppercase filter turns `/Volumes/FILES/Dev` into `/VOLUMES`, `/VOLUMES/FILES`, and `/VOLUMES/FILES/DEV`. section 7.2.4

How Elasticsearch in Action, Second Edition covers it

How deep it goes
Explains component order, index configuration, named filters, and testing, while deferring exhaustive component coverage.
Examples
Uses HTML stripping, uppercasing, Greek-letter extraction, path hierarchies, n-grams, and word shingles.
What it assumes
Assumes basic Elasticsearch text fields and search, while teaching analyzer components and their processing order from the ground up.
What you can do after
Build it from scratch, configure reusable analyzers in index settings, and diagnose token output through the _analyze API.
How it connects
Connects text analysis with character filters, tokenizers, token filters, built-in analyzers, and the _analyze API.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Elasticsearch in Action, Second Edition. The section links point to the parts of the book it draws on.