liveBook · Manning

Agentic RAG

As explained in Designing AI Systems

Agentic RAG is a retrieval-augmented generation approach in which a model decides when and how to retrieve information and can iterate on searches.

Agentic RAG turns retrieval into a model-directed tool rather than a fixed pipeline stage. The model chooses when to search, which index to use, and whether results justify another query. section 10.5

A production-oriented design separates responsibilities:

  • Expose a knowledge-search tool that accepts a focused query and index, then returns document text, identifiers, and relevance scores. section 10.5
  • Let the Data Service handle embeddings, vector search, scoring, and indexed document retrieval while the model evaluates result quality. section 10.5
  • Prepare the index with semantic chunking and metadata so policies, code examples, and organizational documents remain useful retrieval units. section 10.5

Iterative planning research

A user asks for Q2 outcomes, then asks what engineering committed to. If the first search finds revenue data instead of commitments, the model reformulates the query or selects another index and searches again. section 10.5

Listing 10.7 Agentic RAG: the model controls retrieval
def search_knowledge(
    query: str,
    index: str
) -> list:
    results = platform.data.search(
        index_name=index,
        query=query,
        top_k=5,
        score_threshold=0.6
    )
    return [
        {"text": r.text, "source": r.document_id, "score": r.score}    #A
        for r in results    #A
    ]    #A
 
platform.tools.register(
    name="claw.knowledge.search",    #B
    description=(
        "Search the company knowledge base for relevant "
        "information. Use this when the user asks about "
        "company policies, processes, past decisions, or "
        "any organizational information. Formulate a "
        "specific search query based on the conversation "
        "context."
    ),
    parameters={
        "query": {
            "type": "string",
            "description": (
                "A specific search query. Rewrite the "
                "user's question into a search-optimized "
                "query that includes relevant context "
                "from the conversation."
            )
        },
        "index": {
            "type": "string",
            "enum": [
                "company-knowledge",
                "engineering-docs",
                "hr-policies",
                "product-specs"
            ],
            "description": "Which knowledge index to search"
        }
    },
    handler=search_knowledge    
)

Inspect the search-tool interface, including query and index parameters, score thresholds, source identifiers, and relevance scores. section 10.5

How Designing AI Systems covers it

How deep it goes
Explains tool registration, result scoring, query reformulation, and semantic ingestion, but stops short of implementing model reasoning.
Examples
Demonstrates Q2 planning searches, engineering commitments, relevance scores, alternate indexes, and an organizational knowledge index.
What it assumes
Assumes a Data Service and document-ingestion pipeline, while teaching how to expose retrieval and interpret structured results.
What you can do after
Apply it via a registered knowledge-search tool backed by indexed documents and model-controlled iterative retrieval.
How it connects
Connects retrieval-augmented generation with embeddings, vector search, semantic chunking, tool use, and knowledge ingestion.

Also explained in

Compare how 9 books explain agentic RAG →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Designing AI Systems. The section links point to the parts of the book it draws on.