liveBook · Manning

Code coverage

As explained in AI Recommender Systems

Coverage is a recommendation metric measuring the breadth of users, items, or catalog content that a recommender system serves.

For a recommender, coverage measures the breadth of users served and catalog items surfaced. It complements precision and ranking metrics: a model can recommend relevant items near the top while repeatedly showing the same popular items to everyone. section 4.6.4

Check coverage from two directions:

  • User coverage: break down performance, such as hit rate, by user activity level. A gap between power users and cold-start users shows who benefits from personalization and who may receive generic fallbacks. section 4.6.4
  • Item coverage: measure the fraction of the catalog that appears in anyone’s recommendations. Low coverage can signal popularity bias and an invisible long tail. section 4.6.4

A high-ranking model with a narrow catalog

An ALS model recommends only 2.6% of available items, and more than half of those recommended items come from the most popular quartile. Precision or NDCG alone would not reveal how little of the catalog receives exposure. section 4.6.4

Figure 4.18 ALS recommends only a small percentage of available items.

The figure makes the small share of available items recommended by ALS visible. section 4.6.4

Calculate coverage on the final recommendations users see, then inspect it alongside stage-specific results. If coverage falls, check whether retrieval supplies varied candidates before changing the scorer or re-ranker. Compare the result with a popularity baseline and the current model. A coverage loss may justify a re-ranking constraint, but whether that trade-off is worthwhile depends on the product; offline metrics inform the decision rather than settle it. section 4.6.5 section 4.8

How AI Recommender Systems covers it

How deep it goes
Defines user and item coverage, illustrates segment gaps and popularity bias, and applies coverage to pipeline evaluation without specifying calculation code.
Examples
Uses activity-level hit-rate groups, an ALS model recommending 2.6% of the catalog, and a model-versus-baseline comparison.
What it assumes
Assumes familiarity with recommendation lists and metrics such as hit rate and precision while defining user and item coverage directly.
What you can do after
Diagnose/debug with it by separating underserved users or unseen items from ranking failures and checking retrieval before adjusting re-ranking.
How it connects
Connects coverage to hit rate, cold-start users, popularity bias, retrieval, re-ranking, diversity, precision, and NDCG.

Also explained in

Compare how 22 books explain code coverage →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of AI Recommender Systems. The section links point to the parts of the book it draws on.