SVMrank is a learning-to-rank algorithm for search. It converts relevance preferences into binary classification examples, allowing a support vector machine to learn a separating boundary from query, document, and query-document features such as title_bm25 and overview_bm25. section 10.4.1
The transformation begins by normalizing each feature so differences in numeric range do not give one signal undue influence. For every query, SVMrank compares judgments pairwise. When one document has a higher relevance grade, it subtracts the lower-graded document’s feature vector from the higher-graded vector and labels the result +1; the reverse ordering receives −1. The SVM then learns linear weights that reduce out-of-order pairs and generalize to unseen searches. section 10.4.2 section 10.8
Comparing results for “social network”
Suppose The Social Network has normalized features [4.483, 2.100], while Network has [3.101, 1.443]. Because the first document has the higher judgment, create a positive example from [4.483, 2.100] − [3.101, 1.443] with label +1. Create the reversed difference with label −1. Repeating this for every judged document pair produces training data for the SVM. section 10.4.2
Listing 10.9 Transforming features into pairwise data for SVMrank
for doc1_judgment in query_judgments:
for doc2_judgment in query_judgments:
j1_features = numpy.array(doc1_judgment.features)
j2_features = numpy.array(doc2_judgment.features)
if doc1_judgment.grade > doc2_judgment.grade:
predictor_deltas.append(+1) #1
feature_deltas.append(j1_features - #2
j2_features) #2
elif doc1_judgment.grade < doc2_judgment.grade:
predictor_deltas.append(-1) #3
feature_deltas.append(j1_features - #4
j2_features) #4
Inspect the compact loop that compares every pair of judgments, assigns the preference label, and stores the feature difference. section 10.4.2
SVMrank is a practical first model rather than a complete relevance solution. Hold out judgments as a test set to check generalization, then weigh relevance improvements against search-time performance after deployment. section 10.8
