liveBook · Manning

Customer satisfaction

As explained in AI Model Evaluation

Viewing satisfaction is the degree to which users find their consumption of recommended content fulfilling and worthwhile.

Customer satisfaction measures whether recommended content feels fulfilling and worthwhile to consume. For a movie recommender, it is closer to sustained viewing, consumption, and retention than to immediate clicks. A model can increase attention while degrading the experience through disappointing or clickbait-like recommendations. section 8.1.1

Evaluate satisfaction through an outcome-aligned metric framework:

  • Treat CTR as a proxy rather than the goal when the product seeks long-term viewing satisfaction. section 8.1.1
  • Compare clicks with completion, consumption, satisfying sessions, and retention to separate product value from attention. section 8.1.1
  • Version satisfaction metrics as products mature: a later version might combine thumbs-up ratios with surveys or human ratings. section 8.5.2
  • Recheck proxy validity through historical correlations and refresh diagnostics when model behavior or user risks change. section 8.5.2

Novelty-focused movie recommendations

A new model surfaces unseen movies and raises CTR, but users do not finish watching and some stop using the product. The apparent win disappears once satisfaction is evaluated through consumption and retention. If an LLM adds persuasive explanations, also inspect factuality, coherence, novelty, hallucination rate, trust, and long-term satisfaction. section 8.1.1 section 8.5.2

How AI Model Evaluation covers it

How deep it goes
Contrasts weak proxies with outcome measures and metric versioning, but does not specify a complete satisfaction survey or rating protocol.
Examples
Uses novelty-focused movie recommendations, incomplete viewing, product abandonment, metric versions, and LLM explanation diagnostics.
What you can do after
Evaluate recommendation quality by testing whether proxy improvements translate into consumption, retention, satisfaction, and trustworthy user experiences.
How it connects
Connects customer satisfaction to CTR, retention, consumption, proxy metrics, online evaluation, recommender systems, and LLM-generated explanations.

Also explained in

Compare how 5 books explain customer satisfaction →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of AI Model Evaluation. The section links point to the parts of the book it draws on.