A search result clicked once in one examination has a raw click rate of 1.0, but that single observation is weak evidence of relevance. A prior probability represents an initial belief about an unknown quantity before incorporating observations for a particular case. A beta prior can keep a low-confidence relevance grade from being treated as certain. section 11.3.2
Choose a prior relevance grade and a prior weight. For a beta distribution, set prior_a = grade × weight and prior_b = (1 − grade) × weight. Update prior_a with clicks and prior_b with examines minus clicks. The posterior mean, posterior_a / (posterior_a + posterior_b), gives the adjusted relevance grade. A larger prior weight makes observations move the grade more slowly. section 11.3.2
Adjusting a sparse click rate
With a prior grade of 0.3 and weight of 100, the beta parameters are 30 and 70. A result with one click in one examination updates them to 31 and 70, giving an adjusted grade of about 0.307 rather than the raw grade of 1.0. A result with 14 clicks in 34 examinations receives an adjusted grade of about 0.328. section 11.3.2
Listing 11.13 Computing posterior beta distribution
def calculate_sdbn(sessions, prior_grade=0.3, prior_weight=100):
sessions = calculate_prior(sessions, prior_grade, prior_weight)
sessions["posterior_a"] = (sessions["prior_a"] + #1
sessions["clicked"]) #1
sessions["posterior_b"] = (sessions["prior_b"] + #2
sessions["examined"] - sessions["clicked"]) #2
sessions["beta_grade"] = (sessions["posterior_a"] / #3
(sessions["posterior_a"] + sessions["posterior_b"])) #3
return sessions.sort_values("beta_grade", ascending=False)
query = "blue ray"
sessions = get_sample_sessions(query)
bluray_sdbn_data = calculate_sdbn(sessions)
print(bluray_sdbn_data)
The Python calculation updates the beta parameters from clicks and examines, then sorts results by their adjusted grades. section 11.3.2
Confidence weighting preserves sparse examples that a hard examination threshold might discard. It does not eliminate every source of bias: users can act only on results the search system presents, so the resulting feedback still reflects the current ranking. section 11.3.2 section 11.4











