A misspelling is a written query term that differs from its conventional spelling. In search, the usual goal is to normalize the uncommon variant to a canonical form before searching, improving the system’s ability to interpret user intent. section 6.5
User-signal correction treats each query token as a possible spelling variant. It lowercases and trims queries, removes stop words, standalone numbers, and short terms, then counts valid terms. Low-frequency terms become misspelling candidates; high-frequency terms become likely corrections. Quantile thresholds adapt to collections of different sizes, while tighter thresholds favor precision over recall. section 6.5.2
Candidate matching first groups terms by their initial character, then compares spelling candidates with likely corrections using edit distance. Length-sensitive limits allow one edit for shorter words, up to two for medium words, and three for long words. The same process can operate on whole queries to discover multi-term corrections.
From query noise to canonical terms
Observed query signals can produce mappings such as “latop” → “laptop,” “moden” → “modem,” “tosheba” → “toshiba,” and “hp toucpad” → “hp touchpad.” A correction list can then support query rewriting before retrieval. section 6.6
Listing 6.14 Getting all queries searched by users
def get_search_queries():
query = """SELECT searches.user AS user,
LOWER(TRIM(searches.target)) As query #1
FROM signals AS searches WHERE searches.type = 'query'
GROUP BY searches.target, user""" #2
return spark.sql(query).collect()
Start with the SQL query that collects normalized user searches; related listings show token filtering, quantile selection, and correction matching. section 6.5.2
