liveBook · Manning

Query expansion

As explained in AI-Powered Search

Query expansion is a retrieval technique that augments a query with additional terms or representations to improve matching with relevant content.

A search for “advil” can miss relevant documents that use related terms instead. Query expansion augments the original query with additional terms or representations so those documents can match or receive a ranking boost. Sparse lexical expansion uses weighted term-based features rather than relying only on the user's exact words. section 5.4.5 section 5.4.1

Expanding a product query

A semantic knowledge graph (SKG) can generate related terms and weights for a query such as advil: advil OR motrin^0.59897 OR aleve^0.4662 OR ibuprofen^0.3824. The weights let the search request distinguish among the added terms; they do not guarantee that every expanded match is relevant. section 5.4.5

To build an SKG expansion, start from the query term in an indexed field and traverse to related terms. Control the number of returned terms and their minimum occurrences, then use their relatedness scores as weights. The original query can receive a stronger boost than the generated terms. SKG terms come from the index and can be used directly to search document fields without additional dataset fine-tuning. section 5.4.5 section 7.4.3

Listing 5.6 Expanding a query with nodes in an SKG
expansion = ""
for term, stats in traversal["graph"][0]["values"][query] \
                       ["traversals"][0]["values"].items():
  expansion += f'{term}^{stats["relatedness"]} '
expanded_query = f"{query}^5 " + expansion

print(f"Expanded Query:\n{expanded_query}")

This Python listing turns related SKG nodes into weighted terms and gives the original query a boost of 5. section 5.4.5

Choose how expansion affects matching and ranking:

  • Add related terms to the query to allow matches beyond the original wording; a minimum-match setting controls how many terms must match. section 5.4.5
  • Require the original query alongside expansion terms when preserving the original constraint matters, or use expansion terms as query boosts to influence ranking without changing the query itself. section 5.4.5
Listing 5.7 Different query augmentation strategies
def generate_request(query, min_match=None, boost=None):
  request = {"query": query,
             "query_fields": ["title", "body"]}
  if min_match:
    request["min_match"] = min_match
  if boost:
    request["query_boosts"] = boost
  return request

simple_expansion = generate_request(f"{query} {expansion}", "1")
increased_conceptual_precision = \
  generate_request(f"{query} {expansion}", "30%")
increased_precision_same_recall = \
  generate_request(f"{query} AND ({expansion})", "2")
slightly_increased_recall = generate_request(f"{query} {expansion}", "2")
same_results_better_ranking = generate_request(query, "2", expansion)

Compare request configurations for expanded matching, required terms, and ranking-only boosts. section 5.4.5

Context matters when a query has multiple senses. For example, “server” is associated with different terms in development operations and cooking. Query classification and sense disambiguation can be combined with expansion to make search more domain-sensitive. section 6.2 section 5.6

SPLADE-style models provide another route to sparse expansion: a prebuilt language model generates weighted tokens. Unlike SKG output, these may be partial tokens rather than actual indexed terms, so compatibility with the document representation matters. Inspect expansions and compare matching and ranking choices rather than assuming every added term improves relevance. section 7.4.3 section 5.4.5

Listing 7.14 Expanding queries with SPLADE++
from spladerunner import Expander
expander = Expander('Splade_PP_en_v1', 128) #1
queries = ["kimchi", "bbq", "korean bbq",
           "lasagna", "karaoke", "drive through"]

for query in queries:
  sparse_vec = expander.expand(query,  #2
                  outformat="lucene")[0]  #3
  print(sparse_vec)

This SPLADE++ example generates sparse query vectors and returns readable tokens rather than token IDs. section 7.4.3

How AI-Powered Search covers it

How deep it goes
Works through graph traversal, term weighting, and request strategies, then compares graph-based expansion with model-generated sparse tokens.
Examples
Uses weighted medicine terms, a fictional-metal query, contextual senses of “server,” and SPLADE++ food queries.
What you can do after
Apply it via semantic knowledge graph traversal or SPLADE++ and configure expanded matching or ranking boosts.
How it connects
Connects query expansion with semantic knowledge graphs, query classification, query-sense disambiguation, and sparse lexical search.

Also explained in

Compare how 4 books explain query expansion →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of AI-Powered Search. The section links point to the parts of the book it draws on.