A user may search for “cto” while an entity index stores “chief technology officer.” The query parser can match the user's wording, called a surface form, and map it to a canonical form: the chosen representation used to identify that entity. The same mapping can normalize misspellings, acronyms, and alternative labels before search. section 7.4.1 section 6.5
An entity record separates the text to recognize from the interpretation to use:
- surface_form is the spelling or phrase matched in an incoming query; several records can point to one canonical_form. section 7.4.1
- type classifies the match. popularity helps choose among records when one surface form has multiple possible meanings. section 7.4.1
- A canonical form can also identify a semantic function rather than a literal search term: “top,” “best,” and “popular” can map to {popular}. section 7.4.2
From query text to typed entities
For “top kimchi near charlotte,” extraction finds “top,” “near,” and “charlotte.” A typed query tree maps “top” to {popular} and selects {location_distance} for “near”; “kimchi” remains a keyword whose canonical form is “kimchi.” “Charlotte” becomes a city entity. This distinction lets later processing turn “top” into a rating boost and “near” plus the city into a geographic filter, rather than matching those words as ordinary keywords. section 7.4.1 section 7.4.2
Listing 7.8 Generating a typed query tree from a user query
def generate_query_tree(extracted_entities):
query = extracted_entities["query"]
entities = {entity["id"]: entity for entity #1
in extracted_entities["entities"]} #1
query_tree = []
last_end = 0
for tag in extracted_entities["tags"]:
best_entity = entities[tag["ids"][0]] #2
for entity_id in tag["ids"]: #2
if (entities[entity_id]["popularity"] > #2
best_entity["popularity"]): #2
best_entity = entities[entity_id] #2
next_text = query[last_end:tag["startOffset"]].strip()
if next_text:
query_tree.append({"type": "keyword", #3
"surface_form": next_text, #3
"canonical_form": next_text}) #3
query_tree.append(best_entity) #4
last_end = tag["endOffset"]
if last_end < len(query): #5
final_text = query[last_end:len(query)].strip() #5
if final_text: #5
query_tree.append({"type": "keyword", #5
"surface_form": final_text, #5
"canonical_form": final_text}) #5
return query_tree
parsed_query = generate_query_tree(query_entities)
display(parsed_query)
The Python query-tree builder retains unmatched text as keywords and selects the highest-popularity entity for each tagged match, carrying its canonical_form into the tree. section 7.4.1
Canonicalization makes learned alternatives available at query time, but the mapping must reflect the domain. A surface form can still be ambiguous; the illustrated parser uses popularity to select a meaning, while richer interpretation can be added during query enrichment. section 7.4.1 section 7.4.2
