liveBook · Manning

Normalized discounted cumulative gain

As explained in LLM Evaluation and Alignment, The Foundational Ideas

Normalized discounted cumulative gain is a ranking metric that discounts lower-ranked results and normalizes their graded relevance against an ideal ranking.

Normalized discounted cumulative gain (nDCG) evaluates ranked retrieval when relevance has grades such as highly relevant, somewhat relevant, or irrelevant. It discounts relevant documents placed lower in the ranking, then normalizes discounted cumulative gain against the ideal ranking, producing a score from 0 to 1. section 6.2.3

Use nDCG when ranking quality depends on more than the first relevant result:

  • Represent each retrieved document with a graded relevance judgment. section 6.2.3
  • Calculate DCG so lower-ranked relevant documents contribute less. section 6.2.3
  • Normalize DCG by IDCG, the DCG of the ideal ranking, to obtain a value between 0 and 1. section 6.2.3

Choosing among rank-aware metrics

If two relevant documents appear at ranks 3 and 4, nDCG captures how ranking quality changes when relevant material moves lower. Precision@k suits a system that injects a fixed number of documents into context; MRR suits questions where any one good document is sufficient; nDCG suits evaluating all relevant documents with graded relevance, such as a shopping assistant. section 6.2.3

How LLM Evaluation and Alignment, The Foundational Ideas covers it

How deep it goes
Defines rank discounting and ideal-ranking normalization but provides no worked nDCG calculation.
Examples
Contrasts documents at ranks 3 and 4 and recommends nDCG for graded shopping-assistant retrieval.
What you can do after
Evaluate retrieval rankings and choose nDCG when graded relevance across results matters.
How it connects
Connects nDCG to Precision@k, MRR, MAP, graded relevance, and retrieval quality.

Also explained in

Compare how 5 books explain normalized discounted cumulative gain →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of LLM Evaluation and Alignment, The Foundational Ideas. The section links point to the parts of the book it draws on.