liveBook · Manning

Agentic RAG

As explained in LLM Evaluation and Alignment, The Foundational Ideas

An agentic RAG system is a retrieval-augmented generation system that autonomously formulates retrieval queries, evaluates results, and decides whether to continue retrieving.

Agentic RAG extends retrieval-augmented generation with an autonomous reasoning loop. The model formulates retrieval queries, chooses retrieval tools, evaluates results, and decides whether the evidence is sufficient before answering. section 6.1 section 6.1.3

A typical agentic RAG cycle proceeds as follows:

  • Reason about the information gap in the user’s question or in previously retrieved context. section 6.1.3
  • Act by issuing one or more retrieval calls, potentially using different retrieval tools. section 6.1 section 6.1.3
  • Observe the returned documents and incorporate them into the model’s context. section 6.1.3
  • Stop and generate when the evidence is sufficient, or formulate another retrieval query when it is not. section 6.1.3

Evaluation must inspect both the final answer and each retrieve-observe cycle. Faithfulness, answer relevance, context recall, and context precision can be measured per retrieval step and aggregated across the complete trace. Unlike single-turn RAG, evaluation must also detect reasoning gaps in query formulation and retrieval decisions. section 6.1.3 section 6.5

Financial research assistant

A financial assistant receives a question about Q3 revenue growth and Q4 risks. It retrieves earnings-report evidence, answers with supported revenue and cloud-growth claims, and must avoid adding unsupported claims about European expansion, international revenue, or a new CEO. section 6.0

Figure 6.2 Agentic RAG. The AI model decides how to query, which retrieval tool to use, and whether to iterate until it has enough evidence to answer.

The architecture diagram shows the model deciding how to query, which retrieval tool to use, and whether to continue iterating until it has enough evidence. section 6.1

How LLM Evaluation and Alignment, The Foundational Ideas covers it

How deep it goes
Explains the reason-act-observe loop and trace-level evaluation, while stopping short of implementation details for query formulation.
Examples
Uses a financial research assistant and a multi-step retrieval trace as concrete examples.
What it assumes
Assumes familiarity with basic retrieval-augmented generation, retrieval systems, and language-model generation.
What you can do after
Diagnose retrieval, generation, and reasoning failures by evaluating queries, retrieved context, intermediate traces, and final answers.
How it connects
Connects retrieval, generation, information retrieval, hallucination detection, and deep research agents.

Also explained in

Compare how 9 books explain agentic RAG →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of LLM Evaluation and Alignment, The Foundational Ideas. The section links point to the parts of the book it draws on.