liveBook · Manning

Mean average precision

As explained in LLM Evaluation and Alignment, The Foundational Ideas

Mean average precision is the mean of average precision values computed over multiple retrieval queries.

Mean average precision (MAP) evaluates ranking quality across multiple retrieval queries. It averages each query’s average precision, rewarding relevant documents near the top and penalizing relevant documents buried lower in the ranking. section 6.2.3

Calculate MAP in two stages:

  • For each query, compute average precision by averaging precision at the ranks where relevant documents are retrieved. section 6.2.3
  • Average the resulting AP values across all queries to obtain MAP. section 6.2.3
  • Use MAP alongside Precision@k, MRR, or nDCG because each emphasizes a different ranking behavior. section 6.2.3 section 6.5

Two relevant documents at ranks three and four

For a five-document result, precision at rank three is 1/3 and precision at rank four is 2/4. Averaging these values gives AP ≈ 0.417. If the relevant documents move to ranks four and five, AP falls more sharply than Precision@5 alone indicates. MRR for the same ranking is 1/3 ≈ 0.33 because it considers only the first relevant document. section 6.2.3

How LLM Evaluation and Alignment, The Foundational Ideas covers it

How deep it goes
Defines AP and MAP and works through ranked documents, but does not descend into implementation details.
Examples
Uses a five-document ranking with relevant documents at ranks three and four to compare AP, Precision@5, and MRR.
What you can do after
Evaluate retrieval systems by computing per-query AP and averaging those scores across queries.
How it connects
Connects retrieval ranking with Precision@k, Mean Reciprocal Rank, nDCG, context windows, and RAG evaluation.

Also explained in

Compare how 2 books explain mean average precision →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of LLM Evaluation and Alignment, The Foundational Ideas. The section links point to the parts of the book it draws on.