liveBook · Manning

Ragas

As explained in Build an Advanced RAG Application (From Scratch)

RAGAS is an evaluation framework for assessing retrieval-augmented generation systems at the component level, including retrieval and response quality.

Ragas evaluates retrieval-augmented generation pipelines at the component level, using LLM-based, reference-free assessment instead of expensive human-annotated ground-truth data. section 6.5.2

Its core metrics separate retrieval quality from generated-response quality:

  • Context Precision checks whether the most relevant retrieved chunks appear near the top of the results. section 6.5.2
  • Context Recall checks whether retrieved documents cover all relevant aspects of the query. section 6.5.2
  • Faithfulness checks whether the generated response remains factually supported by the retrieved documents. section 6.5.2
  • Answer Relevancy generates artificial questions from the answer and compares them with the original question. section 6.5.2

Comparing RAG configurations

Track the four component metrics separately to identify retrieval or generation weaknesses, then use the composite RAGAS score to compare configurations and monitor improvements over time. section 6.5.2 section 6.6

Figure 6.7 RAGAS Core Metric extensively covers LLM Grounding and Response (Source)

Use the figure to inspect how the core metrics cover retrieved context, LLM grounding, and response quality. section 6.5.2

How Build an Advanced RAG Application (From Scratch) covers it

How deep it goes
Defines four metrics and a composite score, using LLM-based assessment without requiring human-annotated ground truth.
Examples
Uses retrieved chunks, generated responses, artificial questions, and composite scores as evaluation artifacts.
What it assumes
Assumes a RAG pipeline with retrieval and generation stages, while defining evaluation metrics for both stages.
What you can do after
Evaluate RAG configurations and track improvements using component metrics and a composite RAGAS score.
How it connects
Connects retrieval quality with context precision and recall, then links generated responses to faithfulness and answer relevancy.

Also explained in

Compare how 4 books explain Ragas →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Build an Advanced RAG Application (From Scratch). The section links point to the parts of the book it draws on.