liveBook · Manning

Normalized discounted cumulative gain

As explained in Retrieval Augmented Generation, The Foundational Ideas

NDCG@10 is normalized discounted cumulative gain computed over the ten highest-ranked retrieval results.

NDCG@10 is normalized discounted cumulative gain computed over the ten highest-ranked retrieval results. It provides a common basis for comparing retrieval systems. section 7.3

NDCG@10 supports comparative retrieval evaluation:

  • It compares lexical BM25, unsupervised dense Contriever, zero-shot HyDE, and supervised Contriever-FT retrieval. section 7.3
  • Reported scores span TREC DL19, TREC DL20, and BEIR tasks including SciFact and FiQA. section 7.3

Benchmark comparison

On TREC DL19, the listed NDCG@10 scores are BM25 38.3, Contriever 44.5, HyDE 61.9, and Contriever-FT 64.5. section 7.3

Table 7.1 HyDE zero-shot performance vs. baselines
Benchmark
BM25 (Lexical)
Contriever (Unsupervised Dense)
HyDE (Zero-Shot)
Contriever-FT (Supervised Dense)
TREC DL19
38.3
44.5
61.9
64.5
TREC DL20
35.8
43.1
60.1
62.1
BEIR (SciFact)
66.8
64.3
69.7
71.0
BEIR (FiQA)
26.9
30.5
38.8
48.1

Use the benchmark table to compare NDCG@10 across retrieval methods and datasets. section 7.3

How Retrieval Augmented Generation, The Foundational Ideas covers it

How deep it goes
Defines the cutoff and reports comparative scores, but does not explain the metric’s formula or discounting calculation.
Examples
Uses TREC DL19, TREC DL20, BEIR SciFact, and BEIR FiQA comparison scores.
What you can do after
Evaluate retrieval approaches by comparing NDCG@10 scores across benchmarks and baselines.
How it connects
Connects retrieval evaluation with BM25, Contriever, HyDE, supervised dense retrieval, and BEIR benchmarks.

Also explained in

Compare how 5 books explain normalized discounted cumulative gain →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Retrieval Augmented Generation, The Foundational Ideas. The section links point to the parts of the book it draws on.