liveBook · Manning

Similarity search

As explained in AI-Powered Search

A search engine is software that indexes information and retrieves or ranks items in response to user queries.

A query can name a concept without naming every relevant document. Someone searching for “server” might mean restaurant staff or a networked computer. Conceptual search uses relationships among terms, entities, categories, and documents to interpret the intended meaning before matching and ranking results. It complements keyword matching rather than replacing it. section 1.2.4 section 6.2

The searchable content provides a starting point: its terminology, entities, and sentences form a semantic graph of the domain. Query interactions add another source of evidence. For example, keywords and categories in documents returned for a query can help categorize that query. A semantic knowledge graph can then connect a query to likely categories and related terms, supplying context for a rewritten search. section 1.3.4 section 1.3.7

A practical query pipeline has four phases:

  • Parse the query to identify useful phrases, concepts, and entities; account for misspellings when needed. section 6.6 section 7.0
  • Enrich it with likely categories and related terms. Keep ambiguous meanings in separate contexts instead of merging every association into one list. section 6.2
  • Transform that interpretation into a query the search engine can execute, then match and rank the results. These phases can repeat or be skipped as appropriate. section 7.0 section 3.0

Separating senses of “chef”

A semantic-graph traversal can start with the query “chef,” find related categories, and then find terms related to the query within each category. In the supplied results, cooking leads toward “kitchen” and “restaurant,” while devops leads toward “puppet” and “ansible.” The category gives an expansion its meaning: adding both sets of terms indiscriminately would risk mixing two intents. The example uses a context field (`category`), a keyword field (`body`), and limits on the returned contexts and terms. section 6.2

Listing 6.2 Disambiguating query intent across different contexts
def print_query_disambigutaion(query,
      context_field="category", context_limit=5,
      keywords_field="body", keywords_limit=10, min_occurrences=5):

  nodes_to_traverse = [{"field": keywords_field,  #1
                        "values": [query]}, #1
                       {"field": context_field,  #2
                        "min_occurrences": min_occurrences, #2
                        "limit": context_limit}, #2
                       {"field": keywords_field,  #3
                        "min_occurrences": min_occurrences, #3
                        "limit": keywords_limit}] #3

  traversal = skg.traverse(*nodes_to_traverse)
  print_disambigutaions(query, traversal)

The Python traversal shows the query → category → related-terms sequence, with configurable fields, limits, and minimum occurrence counts. section 6.2

Conceptual matching is one part of relevance, not a guarantee of it. Category scores and related terms still need to guide an appropriate query and ranking decision. Content understanding, domain understanding, and user preferences may all contribute to intent; user signals and relevance testing can help determine whether a change actually improves results. Embedding-based vector search and language-model question answering are other semantic-search approaches, not prerequisites for this graph-based pipeline. section 1.2.5 section 1.3 section 7.0

How AI-Powered Search covers it

How deep it goes
Works through a four-phase query pipeline and a configurable graph traversal; treats embedding-based search and question answering separately.
Examples
Contrasts cooking and devops meanings of “chef” through category-specific terms and a Python semantic-graph traversal.
What it assumes
Assumes familiarity with indexed documents, queries, and ranking; explains contextual term discovery through a worked traversal.
What you can do after
Build it from scratch as a query pipeline that contextualizes terms, rewrites queries, and searches ranked results.
How it connects
Connects semantic knowledge graphs, query-sense disambiguation, query expansion, relevance ranking, and personalized search.

Also explained in

Compare how 22 books explain similarity search →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of AI-Powered Search. The section links point to the parts of the book it draws on.