A sidecar collection is an auxiliary search collection stored alongside a primary collection. It holds useful data that supports search, relevance, or ranking without changing the primary documents. section 4.2.2
For signals boosting, a sidecar collection stores aggregated query-document signals. A query lookup retrieves documents users clicked, orders them by click count, and turns those counts into ranking boosts. section 4.2.2
Query-time sidecars provide practical control but introduce operational costs:
- Update one query’s boost document incrementally without rebuilding the primary product collection. section 8.6.1
- Enable or disable boosting by skipping the sidecar lookup or query modification. section 8.6.1
- Swap ranking algorithms and incorporate current signals more easily. section 8.6.1
- Pay for an extra search, added latency, scalability trade-offs, and inefficient or inaccurate pagination. section 8.6.1
Aggregating click signals for “ipad”
Aggregate click events by query and document, then write the results to a signals_boosting collection. A result such as document 885909457588 with a boost of 966 becomes a query boost, helping actual iPads outrank frequently matching accessories. section 4.2.2
Listing 4.6 Creating a signals-boosting model by aggregating signals
signals_collection = engine.get_collection("signals")
create_view_from_collection(signals_collection, "signals") #1
signals_aggregation_query = """
SELECT q.target AS query, c.target AS doc, #2
COUNT(c.target) AS boost #2
FROM signals c LEFT JOIN signals q ON c.query_id = q.query_id
WHERE c.type = 'click' AND q.type = 'query'
GROUP BY q.target, doc #2
ORDER BY boost DESC"""
dataframe = spark.sql(signals_aggregation_query) #3
signals_boosting_collection = \ #4
engine.create_collection("signals_boosting") #4
signals_boosting_collection.write(dataframe) #4
Inspect the SQL aggregation and collection-writing steps used to create a signals-boosting sidecar. section 4.2.2
Listing 4.7 Searching with signals boosting to improve relevance
def search_for_boosts(query, collection, query_field="query"):
boosts_request = {"query": query,
"query_fields": [query_field],
"return_fields": ["query", "doc", "boost"],
"limit": 10,
"order_by": [("boost", "desc")]}
response = collection.search(**boosts_request)
return response["docs"]
def create_boosts_query(boost_documents):
print(f"Boost Documents: \n{boost_documents}")
boosts = " ".join([f'"{b["doc"]}"^{b["boost"]}'
for b in boost_documents])
print(f"\nBoost Query: \n{boosts}\n")
return boosts
query = "ipad"
boost_docs = search_for_boosts(query, signals_boosting_collection)
boosts_query = create_boosts_query(boost_docs)
request = product_search_request(query)
request["query_boosts"] = boosts_query
response = products_collection.search(**request)
display_product_search(query, response["docs"])
Inspect the query-time lookup that converts sidecar records into document boosts for the primary search. section 4.2.2
