liveBook · Manning

Accuracy

As explained in Retrieval Augmented Generation, The Foundational Ideas

Accuracy is the proportion of evaluated system outputs judged correct according to a specified criterion.

Accuracy measures whether evaluated answers are correct under a specified criterion. In retrieval systems, correct answers depend on the retriever finding relevant content; a reader cannot recover an answer from documents it never receives. section 3.2.1

Keep these retrieval measures distinct:

  • Accuracy assesses whether the model answers correctly; retrieval accuracy may instead count whether correct content appears within a specified result set. section 3.2.1 section 15.7.1
  • Recall@k is binary: it counts a question as successful when a correct document appears anywhere in the top k. section 3.2.1
  • MRR rewards higher placement by averaging the reciprocal rank of the first correct result. section 3.2.1

Compare accuracy with operating cost

On an 85-question evaluation set, baseline RAG reached 54.12% retrieval accuracy at 0.79 seconds per query. A-RAG reached 62.35% strict accuracy and 69.41% when semantically equivalent retrievals counted, but latency rose to 5.02 seconds. The gain matters only alongside the added cost. section 15.7.1 section 15.7.3

How Retrieval Augmented Generation, The Foundational Ideas covers it

How deep it goes
Distinguishes answer correctness from Recall@k and MRR, then reports strict and semantic retrieval accuracy.
Examples
Compares baseline RAG at 54.12% Hit Rate@5 with A-RAG at 62.35% strict and 69.41% semantically adjusted accuracy.
What you can do after
Evaluate retrieval systems by comparing accuracy with baselines, semantic scoring, and query latency.
How it connects
Connects retrieval accuracy to Recall@k, MRR, latency, and semantic accuracy.

Also explained in

Compare how 49 books explain accuracy →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Retrieval Augmented Generation, The Foundational Ideas. The section links point to the parts of the book it draws on.