liveBook · Manning

Precision

As explained in AI Recommender Systems

Precision is the proportion of returned items that are relevant to the retrieval or recommendation objective, optionally measured within a specified top-k list.

Precision measures the proportion of returned recommendations that are relevant. Precision@k restricts the calculation to the first k items in a ranked list. section 4.6.1

For one user, precision@k equals the number of relevant items among the top k recommendations divided by k. The chosen k should match the product surface: mobile and desktop layouts can produce different values for the same model. section 4.6.1

A top-10 calculation

After temporal splitting, a user has two relevant test items, 64 and 5. If both appear in the model's top-10 list, precision@10 is 2/10 = 0.2. The corresponding recall@10 is 1.0 because both relevant items were retrieved. section 4.6.1

Listing 4.6 Calculating precision@10
def calculate_precision_at_k(recommendations_df, relevance_set_df, k=10):
    """
    Calculate precision@k for each user.
    Returns DataFrame with [user_id, precision, num_hits]
    """
    top_k_recs = recommendations_df[recommendations_df['rank'] <= k]  #A
    
    merged = top_k_recs.merge(                                         #B
        relevance_set_df,
        on=['user_id', 'item_id'],
        how='left',
        indicator=True
    )
    merged['is_hit'] = (merged['_merge'] == 'both').astype(int)       #C
    
    user_metrics = merged.groupby('user_id').agg(                     #D
        num_hits=('is_hit', 'sum'),
        num_recs=('is_hit', 'count')
    ).reset_index()
    
    user_metrics['precision'] = user_metrics['num_hits'] / user_metrics['num_recs']  #E
    
    return user_metrics
 
als_metrics = calculate_precision_at_k(als_recs, relevance_set, k=10)
 

The implementation filters recommendations by rank, joins them with each user's relevance set, counts hits, and aggregates per-user precision. section 4.4.3

Precision alone does not establish quality. Compare it with random, popularity, or production baselines, then examine ranking position, catalog coverage, and segment differences. Increasing k usually raises recall while lowering precision. section 4.1.2 section 4.4.4

How AI Recommender Systems covers it

How deep it goes
Defines the formula, demonstrates per-user calculation, and connects interpretation to baselines, cutoff choice, and precision-recall trade-offs.
Examples
Uses temporal train-test splitting, a top-10 list containing two relevant items, ALS results, and a popularity baseline.
What you can do after
Evaluate recommender changes against relevance sets and popularity baselines using per-user precision@k.
How it connects
Connects precision with recall, hit rate, popularity baselines, ranking metrics, catalog coverage, and stratified analysis.

Also explained in

Compare how 27 books explain precision →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of AI Recommender Systems. The section links point to the parts of the book it draws on.