liveBook · Manning

Average precision

As explained in AI Recommender Systems

Average precision is the mean of precision values at the ranked positions where relevant items occur for one user or query.

Average precision (AP) summarizes a ranked recommendation list by averaging precision at the positions where relevant items appear. It therefore rewards both relevance and early placement. section 4.6.3

For one user or query, mark items as relevant or irrelevant, scan the ranking, and use each relevant position's precision in the average. Unlike plain precision, AP distinguishes relevant items near the top from equally relevant items placed later. section 4.6.3

Comparing recommendation lists

Two lists can contain the same relevant consumed item while placing it at different positions. AP favors the list that surfaces that item earlier because its precision is measured at the relevant position. section 4.6.3

Listing 4.8 Calculate metrics for different K values
 
for k in k_values: #A
  if k <= len(top_k_indices): #B
    top_k_rel = relevance_scores[:k] #C
    
    if np.sum(top_k_rel) > 0:  #D
      ideal_rel = np.sort(top_k_rel)[::-1] #E
      ndcg_k = ndcg_score([ideal_rel], [top_k_rel])
      all_metrics[k]['ndcg'].append(ndcg_k)
              
    precision_k = np.sum(top_k_rel) / k #F
    all_metrics[k]['precision'].append(precision_k)
              
    if len(test_indices) > 0:
      recall_k = np.sum(top_k_rel) / len(test_indices) #G
      all_metrics[k]['recall'].append(recall_k)
              
    first_relevant = np.where(top_k_rel == 1)[0] #H
    if len(first_relevant) > 0:
      mrr = 1.0 / (first_relevant[0] + 1)
      all_metrics[k]['mrr'].append(mrr)
    else:
      all_metrics[k]['mrr'].append(0.0)
              
    if np.sum(top_k_rel) > 0: #I
      ap = 0.0
      relevant_found = 0
      for i, rel in enumerate(top_k_rel):
        if rel == 1:
        relevant_found += 1
    ap += relevant_found / (i + 1)
    ap /= np.sum(top_k_rel)
     all_metrics[k]['map'].append(ap)
   else:
     all_metrics[k]['map'].append(0.0)

The metric-evaluation listing shows how to slice relevance scores at each K and begins the AP accumulation alongside precision, recall, NDCG, and MRR. section 4.6.3

How AI Recommender Systems covers it

How deep it goes
Defines AP precisely and situates its position-based calculation within a multi-metric evaluation loop, without detailing the full implementation.
Examples
Uses recommendation lists with binary consumed-item relevance and compares identical relevant items placed at different ranks.
What it assumes
Assumes familiarity with ranked lists, relevant items, and precision while using binary consumption as the simplest relevance signal.
What you can do after
Evaluate ranked recommendation outputs with a position-sensitive relevance metric alongside other top-K measures.
How it connects
Connects average precision to precision, recall, mean reciprocal rank, and normalized discounted cumulative gain.

Also explained in

Compare how 2 books explain average precision →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of AI Recommender Systems. The section links point to the parts of the book it draws on.