A RAG assistant can retrieve a clean-looking set of documents and still miss what it needs to answer. Precision@k measures one part of that problem: the fraction of the top k retrieved documents that are relevant to the query. If three of five returned documents are relevant, Precision@5 is 3/5, or 0.6. section 14.1.1 section 8.1
Precision@k asks whether the returned set contains noise. Recall@k asks whether the system found the relevant documents that exist in the corpus. Increasing k may improve coverage, but a longer context can add irrelevant material, cost, and latency. It can also leave useful evidence buried where the LLM pays it less attention. section 8.1
Same precision, different ranking
Consider five results with relevant documents at positions 3, 4, and 5. Move those same documents to positions 1, 2, and 3: Precision@5 remains 0.6. The second ordering is more useful when early evidence matters, but Precision@k cannot distinguish the two. MRR and NDCG@k add sensitivity to rank position. section 14.1.1
Do not select a RAG retriever on Precision@k alone. A narrow result set can score well while omitting another needed aspect of a query. In a jacket search, three similar warm jackets might all count as relevant, while a more varied RAG-Fusion result could receive a lower Precision@3 yet be more useful. Check Recall@k for coverage and consider diversity-aware evaluation when distinct query intents matter. Relevance labels alone do not establish whether the generator used a document. section 8.6 section 14.1.1
Table 14.1 Classical retrieval metrics for RAG evaluation
|
Metric
|
What it measures
|
What it misses in RAG
|
|
Precision@k
|
Fraction of retrieved docs that are relevant
|
Rank position; whether the LLM uses the document
|
|
Recall@k
|
Fraction of relevant docs that were retrieved
|
Whether retrieved docs contain the specific
information needed
|
|
MRR
|
How quickly the first relevant doc appears
|
Quality of the full retrieved set beyond the first hit
|
|
NDCG@k
|
Ranked relevance with position discount
|
Whether "relevant" to the query means
"useful" to the generator
|
|
MAP
|
Average precision across all recall levels
|
Same gap as NDCG: relevance labels don't capture
generation utility
|
Compare what Precision@k captures with the coverage, rank-position, and generation-utility gaps of other retrieval metrics. section 14.1.1


