A ranking model scores search results so that documents likely to satisfy a query appear first. A keyword search engine can rank by term matches; a machine-learned ranking model instead learns how to combine relevance signals. Its score is most useful for ordering documents within the same query, not as an absolute measure of relevance across queries. section 3.0 section 10.4
Learning to rank (LTR) builds a model that can generalize beyond queries with recorded clicks. Given relevance judgments—queries paired with documents labeled as relevant or irrelevant—it learns patterns such as whether strong title matches tend to outrank matches elsewhere. This differs from signals boosting, which depends on interactions with particular query–document pairs. section 4.2.4 section 10.0
A practical LTR pipeline has six parts:
- Collect judgments for query–document pairs, then set some aside for testing. section 10.1.2 section 10.8
- Define features the search engine can compute quickly, such as title-match score, overview-match score, document popularity, or release year. section 10.3 section 10.8
- Log those feature values for judged pairs, train a ranking function, and check whether it orders held-out results well. section 10.3 section 10.8
- Deploy the model and use it to rank results—or, often, rerank only the top results from a faster initial search. section 10.1.2
Movie search: learning feature weights
Suppose a movie search combines matches in title and overview with release year. Manual tuning might assign each feature a boost; LTR learns the weights from judged results instead. For the query “social network,” a relevant movie has a title-match score of 8.243603 and overview-match score of 3.8143613. An irrelevant movie has no title match but an overview-match score of 6.0172443. Those examples give the model evidence that, for this task, a title match may deserve more weight than an overview match. section 10.1.1 section 10.3
Table 10.2 Features logged for the keywords social network for relevant (grade=1) and irrelevant (grade=0) documents
|
Grade
|
Movie
|
title:({keywords})
|
overview: ({keywords})
|
{!func}release_year
|
|---|---|---|---|---|
|
1
|
Social Network
|
8.243603
|
3.8143613
|
2010.0
|
|
0
|
#chicagoGirl
|
0.0
|
6.0172443
|
2013.0
|
|
0
|
Life As We Know It
|
0.0
|
4.353118
|
2010.0
|
|
0
|
The Cheyenne Social Club
|
3.4286604
|
3.1086721
|
1970.0
|
The feature table places relevance grades beside title, overview, and release-year values for one query, making the training inputs and their contrasting matches visible. section 10.3
A simple trained model can normalize each feature, multiply it by a learned coefficient, and add the products to obtain a score. For “wrath of khan,” one worked comparison scores a strong title-and-overview match at 1.702 and a competing movie at −0.371. The ordering, rather than either score alone, is the point. section 10.5.2 section 10.4
A model is not automatically reliable. Test it on judgments withheld from training, and weigh relevance gains against serving speed. Behavioral signals can support regular retraining, but clicks require care: users tend to click higher-ranked results regardless of relevance, and documents with few clicks provide less-confident evidence. section 10.8 section 11.0
