When strings are too long for convenient character-by-character comparison, break them into smaller pieces. An n-gram is a contiguous sequence of n successive tokens, such as characters, substrings, or words. section 8.4
The value of an n-gram comes from comparison: token-based string-similarity methods compare overlapping pieces of longer strings, helping group similar high-cardinality categories or clean noisy spellings. section 8.4
Employee titles
A salary model can encode many employee-position titles by comparing their substring patterns instead of treating every title as unrelated. MinHashEncoder and GapEncoder provide production encoders based on substrings or frequently co-occurring substring combinations. section 8.4
