liveBook · Manning

Stratified analysis

As explained in AI Recommender Systems

Stratified analysis evaluates results separately across predefined subgroups to reveal differences in performance between them.

Stratified evaluation measures recommendation performance separately across user groups, revealing who benefits and who remains underserved. section 4.7

Evaluate across user-lifecycle segments:

Compare segment-level results

A model may improve active-user metrics while worsening cold-start performance. Compare each segment with an appropriate baseline: popularity is a strong benchmark for cold-start users, while power users expect more personalization. section 4.7

Figure 4.19 The different stages of a user's lifecycle, and which types of recommendations can be used at each stage.

Use the lifecycle diagram to relate user stages to recommendation types. section 4.7

How AI Recommender Systems covers it

How deep it goes
Defines segment-wise metrics and illustrates trade-offs and baseline choices, without specifying a statistical procedure.
Examples
Uses cold-start, light, active, and long-term users to show how model performance and trade-offs vary by segment.
What you can do after
Evaluate recommendation quality by user segment, compare baseline expectations, and identify where improvements or trade-offs matter.
How it connects
Connects recommender evaluation to cold-start users, user lifecycle, popularity baselines, and precision@10.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of AI Recommender Systems. The section links point to the parts of the book it draws on.