Top-k evaluation measures a ranked output by inspecting only its first K results. It focuses evaluation on the recommendations users are most likely to see and interact with, rather than treating every ranked position equally. section 1.3.5
Choose the metric according to what matters in the visible recommendation set:
- Precision@K asks how many of the top K recommended items are relevant. section 1.3.5
- Recall@K asks how many relevant items appear within the top K, helping identify missed options. section 1.3.5
- NDCG@K evaluates whether highly relevant items appear higher in the ranked list, making position important. section 1.3.5
- Select K using product context, interpretability, and the amount of the list users realistically inspect; evaluating position 78 may add little value. section 1.3.5
Five-show recommendation list
A system recommends five TV shows, three of which are relevant according to watch history or labeled preferences. Precision@5 is 3/5, or 60%. section 1.3.5
Table 1.2 Example of Offline Metrics for a Recommender Model
|
Metric
|
Metric Category
|
Definition
|
|
Precision at K (P@K)
|
Ranking and Retrieval Metric
|
Of the top K recommended items, how many are relevant? Useful when the quality of the top results matters most.
|
|
Recall at K (R@K)
|
Ranking and Retrieval Metric
|
Of all relevant items, how many appear in the top K? Useful when the system should avoid missing relevant options.
|
|
NDCG@K
|
Ranking Metric
|
Whether highly relevant items appear higher in the ranked list. Useful when position matters.
|
|
Coverage
|
Recommender System Metric
|
How much of the catalog or user base the recommender meaningfully serves. Useful for detecting over-concentration.
|
|
Diversity
|
Recommender System Metric
|
Whether the recommendation set contains varied items rather than many similar ones. Useful for avoiding repetitive experiences.
|
|
Groundedness
|
Generative AI / LLM Metric
|
Whether a generated recommendation explanation is supported by user history, item metadata, or retrieved context.
|
Compare top-k ranking measures with coverage, diversity, and other offline recommender metrics before choosing an evaluation target. section 1.3.5
