liveBook · Manning

Precision@k

As explained in Retrieval Augmented Generation, The Foundational Ideas

Precision@k is the proportion of the top k retrieved results that are relevant to the query.

A RAG assistant can retrieve a clean-looking set of documents and still miss what it needs to answer. Precision@k measures one part of that problem: the fraction of the top k retrieved documents that are relevant to the query. If three of five returned documents are relevant, Precision@5 is 3/5, or 0.6. section 14.1.1 section 8.1

Precision@k asks whether the returned set contains noise. Recall@k asks whether the system found the relevant documents that exist in the corpus. Increasing k may improve coverage, but a longer context can add irrelevant material, cost, and latency. It can also leave useful evidence buried where the LLM pays it less attention. section 8.1

Same precision, different ranking

Consider five results with relevant documents at positions 3, 4, and 5. Move those same documents to positions 1, 2, and 3: Precision@5 remains 0.6. The second ordering is more useful when early evidence matters, but Precision@k cannot distinguish the two. MRR and NDCG@k add sensitivity to rank position. section 14.1.1

Do not select a RAG retriever on Precision@k alone. A narrow result set can score well while omitting another needed aspect of a query. In a jacket search, three similar warm jackets might all count as relevant, while a more varied RAG-Fusion result could receive a lower Precision@3 yet be more useful. Check Recall@k for coverage and consider diversity-aware evaluation when distinct query intents matter. Relevance labels alone do not establish whether the generator used a document. section 8.6 section 14.1.1

Table 14.1 Classical retrieval metrics for RAG evaluation
Metric
What it measures
What it misses in RAG
Precision@k
Fraction of retrieved docs that are relevant
Rank position; whether the LLM uses the document
Recall@k
Fraction of relevant docs that were retrieved
Whether retrieved docs contain the specific information needed
MRR
How quickly the first relevant doc appears
Quality of the full retrieved set beyond the first hit
NDCG@k
Ranked relevance with position discount
Whether "relevant" to the query means "useful" to the generator
MAP
Average precision across all recall levels
Same gap as NDCG: relevance labels don't capture generation utility

Compare what Precision@k captures with the coverage, rank-position, and generation-utility gaps of other retrieval metrics. section 14.1.1

How Retrieval Augmented Generation, The Foundational Ideas covers it

How deep it goes
Defines and calculates the fraction, then identifies its blind spots in ranking and generation utility.
Examples
Uses a five-document compliance query and a jacket search to contrast relevance counts with ranking and diversity.
What it assumes
Assumes retrieved documents can be judged relevant; defines precision and recall through their underlying document sets.
What you can do after
Evaluate/understand it by calculating top-k precision and recognizing when coverage, ranking, or diversity needs another metric.
How it connects
Links Precision@k to Recall@k, RAG-Fusion, MRR, NDCG@k, and intent-aware metrics.

Also explained in

Compare how 3 books explain Precision@k →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Retrieval Augmented Generation, The Foundational Ideas. The section links point to the parts of the book it draws on.