Customer satisfaction measures whether recommended content feels fulfilling and worthwhile to consume. For a movie recommender, it is closer to sustained viewing, consumption, and retention than to immediate clicks. A model can increase attention while degrading the experience through disappointing or clickbait-like recommendations. section 8.1.1
Evaluate satisfaction through an outcome-aligned metric framework:
- Treat CTR as a proxy rather than the goal when the product seeks long-term viewing satisfaction. section 8.1.1
- Compare clicks with completion, consumption, satisfying sessions, and retention to separate product value from attention. section 8.1.1
- Version satisfaction metrics as products mature: a later version might combine thumbs-up ratios with surveys or human ratings. section 8.5.2
- Recheck proxy validity through historical correlations and refresh diagnostics when model behavior or user risks change. section 8.5.2
Novelty-focused movie recommendations
A new model surfaces unseen movies and raises CTR, but users do not finish watching and some stop using the product. The apparent win disappears once satisfaction is evaluated through consumption and retention. If an LLM adds persuasive explanations, also inspect factuality, coherence, novelty, hallucination rate, trust, and long-term satisfaction. section 8.1.1 section 8.5.2




