A search result clicked once after one examination has a raw click grade of 1.0, but that is weak evidence of relevance. A prior distribution represents an initial belief about a result’s relevance before its clicks and examinations are counted. A beta prior lets a click model retain such low-confidence examples while keeping them from dominating learning-to-rank (LTR) training data. section 11.3.2
Set a starting relevance grade, `prior_grade`, and a strength, `prior_weight`. For a grade of 0.3 and weight of 100, the beta prior starts with `prior_a = 30` and `prior_b = 70`. Add observed clicks to `prior_a`; add examinations without clicks to `prior_b`. The adjusted grade is `posterior_a / (posterior_a + posterior_b)`. More observations move the grade farther from its starting value. section 11.3.2
One click versus repeated evidence
For the query “blue ray,” one document has one click from one examination: its raw grade is 1.0, but its beta-adjusted grade is about 0.307. Another has 14 clicks from 34 examinations: its raw grade is about 0.412 and its adjusted grade about 0.328. The latter ranks higher by adjusted grade because its evidence is stronger. section 11.3.2
Listing 11.13 Computing posterior beta distribution
def calculate_sdbn(sessions, prior_grade=0.3, prior_weight=100):
sessions = calculate_prior(sessions, prior_grade, prior_weight)
sessions["posterior_a"] = (sessions["prior_a"] + #1
sessions["clicked"]) #1
sessions["posterior_b"] = (sessions["prior_b"] + #2
sessions["examined"] - sessions["clicked"]) #2
sessions["beta_grade"] = (sessions["posterior_a"] / #3
(sessions["posterior_a"] + sessions["posterior_b"])) #3
return sessions.sort_values("beta_grade", ascending=False)
query = "blue ray"
sessions = get_sample_sessions(query)
bluray_sdbn_data = calculate_sdbn(sessions)
print(bluray_sdbn_data)
The Python calculation shows how clicks and examined-but-unclicked results update the prior, then sorts documents by their adjusted grades. section 11.3.2
Adjusting the prior grade and weight changes how strongly sparse observations are pulled toward the initial belief. These judgments can feed an LTR pipeline, but a prior alone does not address presentation bias: users can click only results they were shown. section 11.3.2 section 11.4
