A search for “advil” can miss a relevant document that says “ibuprofen” but never uses the query word. Query expansion adds related terms or weighted tokens to the search request so documents can match, or rank more highly, without containing the exact query. The original term can retain a stronger boost than its expansions. section 5.4.5 section 5.4.1
One way to generate expansions is to traverse a semantic knowledge graph (SKG) built from indexed content. Start at the query term, retrieve related terms from a document field, and use their relatedness scores as boosts. Because these are terms found in the index, they can be searched directly in document fields without fine-tuning a model to the dataset. section 5.4.5 section 7.4.3
Expanding “vibranium”
In a science-fiction index, a traversal from “vibranium” returns terms including “wakandan,” “adamantium,” and “wakanda.” Python can format the results as a search query such as `vibranium^5 wakandan^0.8197 adamantium^0.80724 wakanda^0.79122`. The stronger boost on the original word keeps an exact match prominent while the related terms provide other paths to relevant documents. section 5.4.5
How the added terms enter the request determines the trade-off:
- Put them alongside the original query to allow more ways to match; adjust `min_match` to require more matching terms when greater conceptual precision is wanted. section 5.4.5
- Require the original query with `AND` while using related terms as additional matching conditions, or place the terms in `query_boosts` to change ranking without changing the requested query. section 5.4.5
- For an ambiguous word, select expansions in context first. “Chef” has different related terms in cooking and DevOps categories; using an unqualified list risks mixing senses. section 6.2
An alternative, SPLADE-style expansion uses a pretrained language model to produce weighted sparse tokens rather than SKG terms taken directly from the index. Some outputs are partial tokens, so using them for search requires compatible indexed tokens. Inspect the generated terms and tune matching and boosts: relatedness alone does not guarantee that every expansion fits the user’s intent. section 7.4.3 section 6.2
Listing 5.7 Different query augmentation strategies
def generate_request(query, min_match=None, boost=None):
request = {"query": query,
"query_fields": ["title", "body"]}
if min_match:
request["min_match"] = min_match
if boost:
request["query_boosts"] = boost
return request
simple_expansion = generate_request(f"{query} {expansion}", "1")
increased_conceptual_precision = \
generate_request(f"{query} {expansion}", "30%")
increased_precision_same_recall = \
generate_request(f"{query} AND ({expansion})", "2")
slightly_increased_recall = generate_request(f"{query} {expansion}", "2")
same_results_better_ranking = generate_request(query, "2", expansion)
The Python request examples show where expanded terms, `min_match`, `AND`, and `query_boosts` enter a search request. section 5.4.5
