liveBook · Manning

Agentic RAG

As explained in Context Engineering

Agentic retrieval-augmented generation uses an AI agent to plan, perform, and evaluate retrieval while producing an answer.

Agentic retrieval-augmented generation (RAG) uses an AI agent to construct context from external knowledge. Rather than following a fixed, one-shot retrieval pipeline, the agent decides what information is needed, selects tools, inspects results, and searches again when evidence is insufficient. It then synthesizes the gathered information into a grounded answer. section 3.5 section 3.1

A typical query-time flow is:

  • Interpret the request and identify the information and tools needed. section 3.5
  • Call retrieval tools over prepared sources, such as a vector database, graph database, web search, or enterprise system. section 3.5
  • Evaluate the accumulated context, refine the search if gaps remain, and generate a response from the collected evidence. section 3.5

Unlike a static hybrid retrieval workflow, agentic RAG can dynamically choose which tools to invoke for a multi-step or ambiguous question. Retrieval still depends on prior indexing or graph construction; the agent orchestrates access to those sources at query time. section 3.4 section 3.5

Shopping comparison

A shopping assistant comparing the battery life of two phones can retrieve specifications from an internal catalog, seek missing details from external sources, and check results for consistency. It aggregates the evidence into a structured comparison while respecting API permissions, latency budgets, and output-format rules. section 3.5

A minimal LangChain implementation can wrap a vector-store retriever as a callable document-search tool. An agent then uses a tool-capable chat model, the tool list, and a system prompt to decide when to search and answer from retrieved passages. section 3.10.5

Listing 3.5 Document retriever wrapped as a tool in an agentic RAG system
from langchain_core.tools import create_retriever_tool

retriever = vector_store.as_retriever() #1
tool = create_retriever_tool( #2
    retriever,
    "search_documents",
    "Search the private document collection, which contains facts
    about books, their authors, and their content.",
)
tools = [tool]

The retriever is exposed as a named tool that the agent can call to search a private document collection. section 3.10.5

Listing 3.6 Tool-calling agent with a local LLM for agentic RAG
from langchain.agents import create_agent
from langchain_ollama import ChatOllama

llm = ChatOllama(model="llama3.1:8b", temperature=0) #1
agent = create_agent( #2
    model=llm,
    tools=tools,
    system_prompt="Answer in one sentence, using only the passages
    returned by the search_documents tool.",
)
question = "Who is the author of the book 'Fake Book: The New Age'?" #3
response = agent.invoke( #4
    {"messages": [{"role": "user", "content": question}]}
)
print(response["messages"][-1].content) #5

The agent uses a local Ollama model and a prompt requiring answers from the search tool’s passages. section 3.10.5

Grounding is not a guarantee of accuracy: source quality, ranking, permissions, and prompt assembly still matter. Choose a more elaborate approach when retrieval evaluation exposes a specific failure that a simpler approach cannot adequately handle. section 3.11 section 3.1

How Context Engineering covers it

How deep it goes
Explains iterative retrieval and source selection, then demonstrates a minimal single-tool agent rather than implementing the full multi-source architecture.
Examples
Illustrates a shopping assistant comparing phone specifications and a private document search for a book’s author.
What you can do after
Apply it via LangChain by wrapping a retriever as a tool and configuring a tool-calling agent.
How it connects
Connects retrieval-augmented generation to tool calling, vector databases, knowledge graphs, and hybrid retrieval.

Also explained in

Compare how 9 books explain agentic RAG →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Context Engineering. The section links point to the parts of the book it draws on.