Click-through rate (CTR) measures how often a displayed search result receives a click: clicks divided by the number of times the result appeared. A result clicked every time has CTR 1; a result never clicked has CTR 0. section 11.1.3
In click modeling, CTR converts search-session behavior into judgments for learning to rank. Group sessions by document, sum its clicks, count the sessions in which it appeared, and divide the two values. Sorting these rates produces a CTR-based ranking. section 11.1.3
A document-level CTR calculation
For the query “transformers dark of the moon,” document 97360810042 has CTR 0.0824, while document 24543750949 has CTR 0.0062. A simple CTR ranking therefore places the former above the latter. section 11.1.3
Listing 11.4 Computing CTR
def calculate_ctr(sessions):
click_counts = sessions.groupby("doc_id")["clicked"].sum()
sess_counts = sessions.groupby("doc_id")["sess_id"].nunique()
ctrs = click_counts / sess_counts
return ctrs.sort_values(ascending=False)
query = "transformers dark of the moon"
sessions = get_sessions(query, index=False)
click_through_rates = calculate_ctr(sessions)
print_series_data(click_through_rates, column="CTR")
Inspect the Python aggregation that sums clicks, counts unique sessions, divides the values, and sorts documents by CTR. section 11.1.3
CTR is not a clean relevance signal because it reflects position bias. Across queries, rank 0 has CTR 0.249727, compared with 0.142673 at rank 1 and lower rates at most later positions. Users click prominent results more often simply because they see them first. Training directly on CTR can reproduce existing rankings instead of improving relevance, so a click model must account for position bias. section 11.2.2
Listing 11.5 CTR per rank in search sessions across all queries
sessions = all_sessions()
num_sessions = len(sessions["sess_id"].unique())
ctr_by_rank = sessions.groupby("rank")["clicked"].sum() / num_sessions
print(ctr_by_rank)
Compare CTR by result rank to quantify how strongly result position affects observed clicks. section 11.2.2
