Training adjusts model weights using data. In search, it can fit a learning-to-rank model to predict relevance; for foundation models, pretraining learns weights from large datasets. section 12.1.2 section 15.1.2
The process depends on the model and goal:
- For learning to rank, select query features, split relevance-judged data into training and test sets, train and upload the model, then evaluate it on held-out data. section 12.1.2
- For Transformer pretraining, text sequences can be used in self-supervised learning that predicts masked tokens. Fine-tuning adds domain data or instructions; prompting supplies input to an already trained model. section 15.1.2
Listing 12.2 Training and evaluating the model on a few features
def train_and_evaluate_model(sessions, model_name, features, log=False):
training_data = generate_training_data(sessions)
train, test = split_training_data(training_data, 0.8)
train_and_upload_model(train, model_name, features=features, log=log)
evaluation = evaluate_model(test, model_name, training_data, log=log)
return evaluation
feature_set = [
ltr.generate_query_feature(feature_name="long_description_bm25",
field_name="long_description"),
ltr.generate_query_feature(feature_name="short_description_constant",
field_name="short_description",
constant_score=True)]
evaluation = train_and_evaluate_model(sessions, "ltr_model_variant_1",
feature_set)
display(evaluation)
The example shows a learning-to-rank workflow using long- and short-description query features, an 80/20 data split, model training, and evaluation. section 12.1.2
Held-out evaluation helps check whether a ranking model learned beyond its training examples, but live-user performance may differ; an A/B test can reveal that gap. section 12.1.2
