Precision measures how much of a selected result set is relevant. In search ranking, top-k precision counts relevant results among the first k results for each query, then averages those proportions across queries. section 10.5.3
For learning-to-rank evaluation, keep queries—not individual judgments—separate between training and testing. Rank each test query with the trained model, count relevant results in the top five, and average the query scores. This tests generalization rather than memorization. section 10.5.3
Precision also supports structured evaluation of generative systems:
- Compare extracted model entities with human-labeled entities, counting true positives, false positives, and false negatives before calculating the metric. section 15.2.5
- Use a golden set to provide corrected reference labels for passages and evaluate whether generated person, organization, and location entities are accurate. section 15.2.5
Ranking the top five
If a ranked query result contains three relevant documents among its first five positions, its precision is 3/5, or 0.6. Averaging this value across test queries produces the model-level score; an initial example reaches approximately 0.36. section 10.5.3
