Click-through rate (CTR) measures how many users click a displayed result or link. In search and recommendation systems, it provides a faster behavioral signal than purchases because clicks occur more often. section 5.2.2 section 5.3.2
Use CTR as part of a metric chain, not as the final business objective:
- Treat CTR as an online metric measured while the system is live. section 5.3.2
- Use it as a proxy for conversion rate when clicks are positively related to purchases. section 5.2.2
- Exploit its greater volume to obtain more training examples and faster, more sensitive A/B tests. section 5.2.2
- Check that optimizing CTR remains connected to purchases and broader outcomes such as revenue. section 5.2.2
Search results at PhotoStock
A search engine expects higher-ranked images to receive more clicks, so CTR evaluates how often users click its result links. Conversion rate separately evaluates which clicks lead to purchases; ranking metrics such as mean reciprocal rank and normalized discounted cumulative gain can provide offline proxies. section 5.3.2
Table 5.1 Examples of offline and online metrics
|
Offline metrics
|
Online metrics
|
|---|---|
|
Recall at given specificity for spam message classification
|
Number of user complaints about spam messages
|
|
Quantiles of 1.5, 25, 50, 75, 95, and 99
|
Value of expired items, total sales
|
|
Mean reciprocal rank, normalized discounted cumulative gain
|
Click-through rate on search engine result page
|
Compare CTR with offline ranking measures and other online business outcomes. section 5.2.2
