liveBook · Manning

Diversity

As explained in AI Recommender Systems

Diversity measures the variety among items within a recommendation list, including differences in genres or other item characteristics.

Diversity measures variety in a single user's recommendation list. Ten highly relevant action movies from the same franchise might achieve high precision while offering little choice. A list spanning different facets of the user's interests can be more useful, even if some items are slightly less relevant. section 4.6.4 section 4.1.3

Several measures capture different kinds of variety:

  • Intra-list diversity averages the dissimilarity between pairs of recommended items. section 4.6.4
  • Category coverage counts the categories represented in each list. section 4.6.4
  • Aggregate diversity counts unique items recommended across all users; unlike the first two measures, it describes system-wide output rather than one user's list. section 4.6.4

Measure diversity at the final output, but diagnose it throughout the pipeline. Retrieval must supply varied candidates: a scorer cannot rank items it never receives, and a final re-ranker may have to make costly corrections if the candidate pool is narrow. Retrieving more candidates than the final list needs gives ranking room to create variety after filtering. section 4.6.5 section 2.2.5

A diversity constraint in final re-ranking may replace a highly relevant item with a less relevant but more varied one, lowering precision. Compare diversity with accuracy and other end-to-end metrics rather than treating either score as the whole decision. Item metadata also has a limit: differences it can measure may not match what users perceive as varied. section 4.6.5 section 4.6.4

How AI Recommender Systems covers it

How deep it goes
Defines three diversity measures and explains pipeline trade-offs, but defers strategies for balancing diversity and relevance.
Examples
Contrasts ten similar action movies from one franchise with a list covering different facets of a user's interests.
What it assumes
Builds on recommendation lists and relevance, while defining diversity and its measures directly.
What you can do after
Diagnose/debug with it by comparing final-list variety with precision and checking whether retrieval supplies sufficiently varied candidates.
How it connects
Connects diversity to precision, category coverage, item coverage, candidate retrieval, and re-ranking.

Also explained in

Compare how 10 books explain diversity →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of AI Recommender Systems. The section links point to the parts of the book it draws on.