Irrelevant retrieved passages use context-window space and can distract a generation model. Context precision measures whether passages that contribute to the ground truth answer appear early in the retrieved context. It is a retrieval-quality metric, not a measure of whether the generated answer is faithful. section 6.3.4 section 6.3
For each retrieved passage at rank k, assign a relevance label vₖ: 1 if it contributes information to the ground truth answer, otherwise 0. Calculate precision@k as the fraction of passages through rank k that are relevant. Context precision is the sum of precision@k × vₖ, divided by the number of relevant passages in the evaluated context. Earlier relevant passages receive more credit. Unlike standard average precision, this calculation uses the number of relevant passages retrieved, not all known relevant documents in the corpus. section 6.3.4
Relevant passages ranked late
Suppose five financial research passages have relevance labels 0, 0, 1, 1, 0. The relevant passages are at ranks 3 and 4, with precision@k values of 1/3 and 2/4. Context precision is (0.333 + 0.500) / 2 = 0.417. section 6.3.4
Listing 6.4 Computing RAGAS context precision
class Relevance(BaseModel):
relevant: Literal[‘RELEVANT’, ‘IRRELEVANT’]
CONTEXT_RELEVANCE_PROMPT = """ Given a QUESTION and a CONTEXT PASSAGE, determine whether
the context passage contains information relevant to answering the question.
QUESTION: {question}
CONTEXT PASSAGE:
{passage}
Respond with exactly one of: RELEVANT, IRRELEVANT"""
def context_precision(question: str, retrieved_passages: list[str],
model: str = "gpt-5.4-mini") -> dict:
"""Compute RAGAS context precision for retrieved passages."""
relevance_labels = []
for passage in retrieved_passages:
response = client.chat.completions.parse(
model=model,
messages=[{
"role": "user",
"content": CONTEXT_RELEVANCE_PROMPT.format(
question=question,
passage=passage
)
}],
temperature=0,
max_tokens=10
)
verdict = response.choices[0].message.parsed
relevance_labels.append(verdict.relevant == "RELEVANT")
# Compute average precision
precision_at_k_sum = 0.0
relevant_so_far = 0
for i, is_relevant in enumerate(relevance_labels, start=1):
if is_relevant:
relevant_so_far += 1
precision_at_k_sum += relevant_so_far / i #A
total_relevant = sum(relevance_labels)
avg_precision = precision_at_k_sum / total_relevant if total_relevant > 0 else 0.0
return {
"context_precision": avg_precision,
"relevant_passages": total_relevant,
"total_passages": len(retrieved_passages),
"relevance_labels": list(zip(retrieved_passages, relevance_labels))
}
The listing illustrates passage-by-passage relevance labeling with a model. Its prompt asks whether a passage is relevant to the question; that judgment is not identical to checking whether it contributes to the ground truth answer. section 6.3.4
Interpret the score alongside context recall and faithfulness. High precision can coexist with low recall when retrieved passages are useful but important information is missing. It can also coexist with low faithfulness when the answer fabricates claims despite relevant context. For low precision, investigate the retrieval method: tune a dense-retrieval similarity threshold, reduce the number of passages, review broad BM25 query terms, or adjust hybrid retrieval weighting. section 6.3 section 6.4.1

