liveBook · Manning

Hit rate

As explained in AI Recommender Systems

Hit rate is a binary metric equal to one when at least one relevant item appears among the top-k recommendations and zero otherwise.

Hit rate is a binary top-k metric: it equals 1 when at least one relevant item appears in the recommendation list and 0 otherwise. section 4.6.1

Use hit rate to answer whether a recommendation list surfaced anything relevant.

  • Prefer it when users typically engage with only one item per session; knowing that any relevant item appeared can matter more than counting every match. section 4.6.1
  • Interpret it alongside precision and recall, since increasing k generally raises the chance of a hit while also adding irrelevant recommendations. section 4.6.1
  • Compare it against random, popularity, and simple personalization baselines rather than treating the score as meaningful in isolation. section 4.10

Single-item session

For a session whose test set contains one relevant item, assign hit rate@k = 1 if that item appears anywhere in the top-k recommendations; otherwise assign 0. section 4.6.1

Aggregate hit rate can hide unequal user experiences, so inspect its distribution across users and evaluate it with ranking, coverage, and diversity metrics in the full pipeline. section 4.10 section 4.8

How AI Recommender Systems covers it

How deep it goes
Defines binary top-k calculation and contrasts it with precision and recall, but does not detail implementation code.
Examples
Uses single-relevant-item sessions and compares hit rate with accuracy, ranking, coverage, and diversity metrics.
What it assumes
Assumes top-k recommendations and relevance sets constructed from historical test data.
What you can do after
Evaluate whether a recommender places at least one relevant item in the top-k and compare results with baselines.
How it connects
Connects to precision, recall, NDCG, MRR, MAP, baselines, and multi-stage pipeline evaluation.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of AI Recommender Systems. The section links point to the parts of the book it draws on.