liveBook · Manning

Canonical form

As explained in AI-Powered Search

A canonical representation is a normalized form that consistently identifies an entity or concept independently of its alternative textual expressions.

A search for “cyber security” should be able to find the same concept as “cybersecurity.” A canonical form is the chosen representation that lets alternative expressions map to one term or entity. Normalizing a less common spelling to a canonical form allows the search to use that form consistently. section 6.5 section 7.4.1

An entity collection stores each matchable expression as a surface_form and its chosen representation as a canonical_form. The two are often identical, but several surface forms can point to the same canonical form. The mapping can handle misspellings, such as “amin” to “admin,” and acronyms, such as “cto” to “chief technology officer.” section 7.4.1

A surface form does not always identify one meaning. For example, “cto” can also mean “cancelled-to-order.” Separate entries can map the same surface form to different canonical forms; the query-tree implementation selects the candidate with the highest popularity value. section 7.4.1

Canonical forms in a query tree

For “top kimchi near charlotte,” extraction maps “top” to {popular} and “near” to {location_distance}; the unrecognized keyword “kimchi” keeps its own text as its canonical form. The selected “near” entity carries a semantic function that can interpret a following city as a geographic filter. Canonical forms can therefore identify interpretation logic as well as normalized terms. section 7.4.1 section 7.4.2

Listing 7.8 Generating a typed query tree from a user query
def generate_query_tree(extracted_entities):
  query = extracted_entities["query"]
  entities = {entity["id"]: entity for entity  #1
              in extracted_entities["entities"]}  #1
  query_tree = []
  last_end = 0

  for tag in extracted_entities["tags"]:
    best_entity = entities[tag["ids"][0]]  #2
    for entity_id in tag["ids"]:  #2
      if (entities[entity_id]["popularity"] >  #2
          best_entity["popularity"]):  #2
        best_entity = entities[entity_id] #2

    next_text = query[last_end:tag["startOffset"]].strip()
    if next_text:
      query_tree.append({"type": "keyword",  #3
                         "surface_form": next_text,  #3
                         "canonical_form": next_text})  #3
    query_tree.append(best_entity)  #4
    last_end = tag["endOffset"] 

  if last_end < len(query):  #5
    final_text = query[last_end:len(query)].strip()  #5
    if final_text:  #5
      query_tree.append({"type": "keyword",  #5
                         "surface_form": final_text,  #5
                         "canonical_form": final_text})  #5
  return query_tree

parsed_query = generate_query_tree(query_entities)
display(parsed_query)

Inspect how extracted tags become query-tree nodes and how popularity selects among candidate entities. section 7.4.1

The mappings depend on the entries available to the extractor; unknown keywords remain keywords in the query tree. Alternative representations can be added to the entity collection as they are learned, while enrichment can attach other domain signals to parsed entities. section 7.4.1 section 7.4.2

How AI-Powered Search covers it

How deep it goes
Explains entity fields, ambiguous matches, extraction, and query-tree selection with an implementation, while leaving broader enrichment extensions open.
Examples
Illustrates mappings with misspellings and acronyms, then parses a restaurant query containing popularity and location expressions.
What you can do after
Apply entity mappings via Solr's text tagger and a Python query-tree implementation.
How it connects
Connects canonical forms to entity extraction, spelling correction, query parsing, and semantic functions.

Also explained in

Compare how 2 books explain canonical form →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of AI-Powered Search. The section links point to the parts of the book it draws on.