A search for “cyber security” should be able to find the same concept as “cybersecurity.” A canonical form is the chosen representation that lets alternative expressions map to one term or entity. Normalizing a less common spelling to a canonical form allows the search to use that form consistently. section 6.5 section 7.4.1
An entity collection stores each matchable expression as a surface_form and its chosen representation as a canonical_form. The two are often identical, but several surface forms can point to the same canonical form. The mapping can handle misspellings, such as “amin” to “admin,” and acronyms, such as “cto” to “chief technology officer.” section 7.4.1
A surface form does not always identify one meaning. For example, “cto” can also mean “cancelled-to-order.” Separate entries can map the same surface form to different canonical forms; the query-tree implementation selects the candidate with the highest popularity value. section 7.4.1
Canonical forms in a query tree
For “top kimchi near charlotte,” extraction maps “top” to {popular} and “near” to {location_distance}; the unrecognized keyword “kimchi” keeps its own text as its canonical form. The selected “near” entity carries a semantic function that can interpret a following city as a geographic filter. Canonical forms can therefore identify interpretation logic as well as normalized terms. section 7.4.1 section 7.4.2
Listing 7.8 Generating a typed query tree from a user query
def generate_query_tree(extracted_entities):
query = extracted_entities["query"]
entities = {entity["id"]: entity for entity #1
in extracted_entities["entities"]} #1
query_tree = []
last_end = 0
for tag in extracted_entities["tags"]:
best_entity = entities[tag["ids"][0]] #2
for entity_id in tag["ids"]: #2
if (entities[entity_id]["popularity"] > #2
best_entity["popularity"]): #2
best_entity = entities[entity_id] #2
next_text = query[last_end:tag["startOffset"]].strip()
if next_text:
query_tree.append({"type": "keyword", #3
"surface_form": next_text, #3
"canonical_form": next_text}) #3
query_tree.append(best_entity) #4
last_end = tag["endOffset"]
if last_end < len(query): #5
final_text = query[last_end:len(query)].strip() #5
if final_text: #5
query_tree.append({"type": "keyword", #5
"surface_form": final_text, #5
"canonical_form": final_text}) #5
return query_tree
parsed_query = generate_query_tree(query_entities)
display(parsed_query)
Inspect how extracted tags become query-tree nodes and how popularity selects among candidate entities. section 7.4.1
The mappings depend on the entries available to the extractor; unknown keywords remain keywords in the query tree. Alternative representations can be added to the entity collection as they are learned, while enrichment can attach other domain signals to parsed entities. section 7.4.1 section 7.4.2

