liveBook · Manning

Content discovery rate

As explained in AI Model Evaluation

Content discovery rate measures how many new titles or genres users try within a defined period.

Content discovery rate measures how many new titles or genres each user tries during a defined period. In recommendation experiments, it complements click-through rate and watch time by showing whether users explore beyond familiar content. section 6.3.6

Use the metric as part of an A/B-test evaluation rather than as a standalone quality signal:

  • Measure new titles or genres tried per user per week. section 6.3.6
  • Pair it with primary metrics such as recommendation click-through rate and average watch time per session. section 6.3.6
  • Check guardrails including latency, playback starts, satisfaction signals, error rates, and retention-sensitive metrics. section 6.3.6

Movie recommendation experiment

Compare the current recommendation model with variants that add contextual features. Track whether users try more new titles or genres, while accepting a treatment only when primary metrics improve without meaningful guardrail degradation. section 6.3.6

How AI Model Evaluation covers it

How deep it goes
Defines weekly measurement and situates it among primary metrics and guardrails, without detailing calculation or statistical analysis.
Examples
Uses a movie recommendation experiment comparing a production model with contextual-feature treatments.
What you can do after
Evaluate recommendation changes by balancing increased content exploration against primary-metric gains and operational guardrails.
How it connects
Connects content discovery rate with recommendation systems, click-through rate, watch time, A/B testing, and guardrail metrics.

This explainer was generated by Manning's knowledge graph from the text of AI Model Evaluation. The section links point to the parts of the book it draws on.