Active learning makes a search-ranking system choose which results to expose for feedback, rather than learning only from clicks on results it already returns. It addresses presentation bias: users cannot click documents they never see, creating a feedback loop that limits the model's knowledge. section 12.0 section 12.3
A practical active-learning loop combines exploitation with deliberate exploration:
- Exploit the current learning-to-rank model by showing results already supported by interaction data. section 12.3
- Explore plausible but underexposed documents, even when they have no existing click data. section 12.3
- Collect user signals from the exploration traffic and regenerate judgments or training data. section 12.3.3
- Retrain and evaluate the ranking model, then repeat exploration across queries and candidates. section 12.3.3
Exploring “transformers dvd”
A ranking system explores results for the query “transformers dvd” that lack reliable click evidence, generates new simulated sessions, and rebuilds training judgments. The resulting model returns different products and wins a simulated A/B test, receiving 145 outcomes against 21 for the earlier model. section 12.3.3
Figure 12.1 Presentation bias’s negative feedback loop. Users never click on what the search engine never returns, so relevance models can never grow beyond the current model’s knowledge.
Use the feedback-loop diagram to see why click-trained ranking cannot discover results hidden by the current ranking. section 12.0
