Agentic retrieval-augmented generation (RAG) uses an AI agent to construct context from external knowledge. Rather than following a fixed, one-shot retrieval pipeline, the agent decides what information is needed, selects tools, inspects results, and searches again when evidence is insufficient. It then synthesizes the gathered information into a grounded answer. section 3.5 section 3.1
A typical query-time flow is:
- Interpret the request and identify the information and tools needed. section 3.5
- Call retrieval tools over prepared sources, such as a vector database, graph database, web search, or enterprise system. section 3.5
- Evaluate the accumulated context, refine the search if gaps remain, and generate a response from the collected evidence. section 3.5
Unlike a static hybrid retrieval workflow, agentic RAG can dynamically choose which tools to invoke for a multi-step or ambiguous question. Retrieval still depends on prior indexing or graph construction; the agent orchestrates access to those sources at query time. section 3.4 section 3.5
Shopping comparison
A shopping assistant comparing the battery life of two phones can retrieve specifications from an internal catalog, seek missing details from external sources, and check results for consistency. It aggregates the evidence into a structured comparison while respecting API permissions, latency budgets, and output-format rules. section 3.5
A minimal LangChain implementation can wrap a vector-store retriever as a callable document-search tool. An agent then uses a tool-capable chat model, the tool list, and a system prompt to decide when to search and answer from retrieved passages. section 3.10.5
Listing 3.5 Document retriever wrapped as a tool in an agentic RAG system
from langchain_core.tools import create_retriever_tool
retriever = vector_store.as_retriever() #1
tool = create_retriever_tool( #2
retriever,
"search_documents",
"Search the private document collection, which contains facts
about books, their authors, and their content.",
)
tools = [tool]
The retriever is exposed as a named tool that the agent can call to search a private document collection. section 3.10.5
Listing 3.6 Tool-calling agent with a local LLM for agentic RAG
from langchain.agents import create_agent
from langchain_ollama import ChatOllama
llm = ChatOllama(model="llama3.1:8b", temperature=0) #1
agent = create_agent( #2
model=llm,
tools=tools,
system_prompt="Answer in one sentence, using only the passages
returned by the search_documents tool.",
)
question = "Who is the author of the book 'Fake Book: The New Age'?" #3
response = agent.invoke( #4
{"messages": [{"role": "user", "content": question}]}
)
print(response["messages"][-1].content) #5
The agent uses a local Ollama model and a prompt requiring answers from the search tool’s passages. section 3.10.5
Grounding is not a guarantee of accuracy: source quality, ranking, permissions, and prompt assembly still matter. Choose a more elaborate approach when retrieval evaluation exposes a specific failure that a simpler approach cannot adequately handle. section 3.11 section 3.1








