Context recall measures how many relevant information pieces a retrieval system captures for answering a query. It evaluates whether each answer sentence is supported by the retrieved context, using binary Yes/No judgments. section 8.2.1 section 8.4
Use context recall to:
- Compare retrieved contexts with ground-truth answers in a benchmark. section 8.2.5
- Identify retrieval gaps that may limit answer correctness. section 8.2.6
Benchmark result
A context recall score of 0.7941 indicates decent retrieval coverage but shows that some necessary information is still missed. Improving retrieval, query reformulation, or entity mapping may raise answer accuracy. section 8.2.6
Listing 8.2 Generating answers and ground truth responses
answers = []
ground_truths = []
latencies = []
contexts = []
for i, row in tqdm(test_data.iterrows(), total=len(test_data), desc="Processing rows"):
ground_truth, _, _ = neo4j_driver.execute_query(row["cypher"]) #1
ground_truths.append([str(el.data()) for el in ground_truth])
start = datetime.now()
try:
answer, context = get_answer(row["question"]) #2
context = [el['content'] for el in context]
except Exception:
answer, context = None, []
latencies.append((datetime.now() - start).total_seconds()) #3
answers.append(answer)
contexts.append(context)
#4
test_data['ground_truth'] = [str(el) for el in ground_truths]
test_data['answer'] = answers
test_data['latency'] = latencies
test_data['retrieved_contexts'] = contexts
The evaluation workflow records answers, ground truths, latency, and retrieved contexts before scoring them with RAGAS. section 8.2.5
