liveBook · Manning

Click-through rate

As explained in AI Model Evaluation

Click-through rate is the proportion of users or impressions that result in a click on a presented item or recommendation.

Click-through rate (CTR) measures how often a presented recommendation earns a click. In a movie recommendation page, it is the percentage of users who click at least one recommendation after the page loads. CTR connects technical performance to user engagement, but it should be read alongside downstream consumption and other product outcomes. section 4.4.3

Use CTR as one part of an evaluation anatomy:

  • Pair it with system metrics such as p50, p95, and p99 page-load time to connect latency with behavior. section 4.4.3
  • Compare it with bounce rate, abandonment rate, and downstream consumption rather than treating a click as the final outcome. section 4.4.3
  • Segment experiments by device type because latency and engagement effects can differ across desktop, iOS, and Android. section 4.4.3
  • Revalidate CTR as a proxy when models or products change; generated explanations can make clicks persuasive without making recommendations satisfying. section 8.5.2

Latency degradation in movie recommendations

A mobile experiment compared a control against five added-latency variants. CTR fell from 23.5% with no added latency to 8.7% with 500 milliseconds added, while average page-load time rose from 1050 to 1552 milliseconds and bounce rate increased from 27.8% to 56.9%. The result links slower delivery with weaker immediate engagement, but downstream viewing is still needed to assess recommendation value. section 4.4.3

How AI Model Evaluation covers it

How deep it goes
Defines CTR and applies it in a segmented latency experiment, but stops short of formal estimation or statistical testing.
Examples
Uses a movie recommendation page, device-segmented latency variants, and mobile results ranging from 23.5% to 8.7% CTR.
What it assumes
Assumes familiarity with recommendation pages and A/B tests, while defining CTR and distinguishing it from related user metrics.
What you can do after
Evaluate recommendation engagement through CTR while checking latency, downstream consumption, and longer-term satisfaction.
How it connects
Connects CTR with A/B testing, latency, bounce rate, abandonment rate, downstream consumption, and recommendation systems.

Also explained in

Compare how 5 books explain click-through rate →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of AI Model Evaluation. The section links point to the parts of the book it draws on.