A photo search result can contain relevant images yet still frustrate users if the best ones appear too far down the page. Normalized discounted cumulative gain (NDCG) measures the quality of a ranked result list using relevance scores, giving greater weight to results near the top. It compares the list’s discounted cumulative gain with the ideal ordering’s gain. section 5.3.2
NDCG is a fast offline metric: it can help assess a search change before that change reaches users. It is not a direct measure of clicks or purchases. Calculating it requires relevance judgments for query results, and those judgments should represent results users are likely to encounter. section 5.3.2
Two ways to score photo-search results illustrate the choice of relevance data:
- NDCG@10 uses human-labeled relevance scores for photos on a search results page. section 12.4.2
- NDCG@30 uses implicit feedback: a purchase scores 3, a favorite 2, a click 1, and a view without interaction 0. section 12.4.2
Before treating an NDCG improvement as a business win, test whether the offline metric tracks an online outcome. One approach is to score results from the existing search algorithm, compare offline metrics with click-through and conversion rates over time, and select metrics with the strongest rank correlation. A/B tests can then provide evidence about how offline differences relate to online changes. section 12.4.2 section 5.3.2
