concept signal in category AI

appears as: signals, signal, signals, The signals
AI-Powered Search

This is an excerpt from Manning's book AI-Powered Search.

Signals are one of the two kinds of fuel that power the intelligence engine of your AI-powered search application, with the other type being your content.

Listing 4.4. Indexing the User Signals Dataset
#Create Signals Collection
signals_collection="signals"
create_collection(signals_collection)

print("Loading Signals...")
csvFile = "../data/retrotech/signals.csv"
signals_update_opts={"zkhost": "aips-zk", "collection": signals_collection, "gen_uniq_key": "true", "commit_within": "5000"}
csvDF = spark.read.format("com.databricks.spark.csv").option("header", "true").option("inferSchema", "true").load(csvFile)
csvDF.write.format("solr").options(**signals_update_opts).mode("overwrite").save()
print("Signals Schema: ")
csvDF.printSchema()
print("Status: Success")

In the last section, we covered how to capture signals from users as they interaction with your search engine. While the signals themselves are useful to help us understand how our search engine is being used, they also serve as the primary inputs for building models that can continually learn from user interactions and enable your search engine to self-tune its relevance model. In this section we’ll introduce how these self-tuning model work through the concept of Reflected Intelligence.

Figure 4.4 demonstrates how how signals boosting works as a continuous feedback loop.

Figure 4.4. Signals Boosting Feedback Loop. A user’s search is logged, and the current signals boosting model is applied to return boosted results. After users take action on those results, the signals from all user interactions with documents are aggregated by originating query to genrate an updated signals boosting model to further improve future searches.
signals boosting model

Signals boosting is one of the simplest forms of Reflected Intelligence, but also one of the most impactful for improving relevance of your most popular, highest-traffic queries. Once your products are indexed and you’ve started collecting signals for your users' queries and document interactions, the only additional steps necessary for implementing signals boosting are to aggregate your signals, and then to add your aggregated signals as boosts to either your queries or your documents. Listing 4.6 demonstrates a simple model for aggregating signals into a side-car collection.