Chapter 11: Tips for building agentic systems

Introduction

Introduces the chapter’s production-informed tips and previews guidance for customer support, RAG, and deep research agents.

11.1 Field-tested tips organized by the five agentic layers

This section introduces the five agentic layers as the framework for the practical tips that follow.

11.1.1 The core layer: Persona

This section gives four practical guidelines for designing an agent persona that shapes behavior throughout the agent.

• Persona as behavioral contract: Explain why the persona should be treated as an API contract rather than merely descriptive prose.
  Why it matters: The persona influences more than the agent’s tone; it helps govern how the agent behaves.
• Focused role and boundaries: Describe how to set a narrow agent role with explicit boundaries and an uncertainty off-ramp, and why that focus helps.
  Why it matters: Clear limits make the agent’s behavior more dependable and its performance easier to assess.
• Schema matched structured responses: Explain when to prefer structured output and what benefit a schema-matched JSON response provides.
  Why it matters: Predictable output helps other parts of a system use the agent’s responses reliably.
• Relevant runtime instruction facts: Describe how to supply runtime facts through dynamic instructions without overinjecting them.
  Why it matters: Keeping runtime context relevant gives the agent useful current information without overwhelming its instructions.

Key terms: persona, API contract, prompt instructions, role, boundaries, I don’t know off-ramp, single-responsibility agents, structured output, JSON, schema

11.1.2 Tools and agent actions

This section gives practical guidelines for choosing, designing, controlling, and maintaining agent tools without burdening the agent with unnecessary tools.

• Focused tool design: Explain why tools should have a single responsibility, clear names and descriptions, and typed inputs, and why function tools are useful for code and APIs.
  Why it matters: A clear tool interface helps the agent choose and use the tool appropriately.
• Tool set size and overhead: Justify keeping an agent’s tool set small and focused on its role, including when tools come from an MCP server.
  Why it matters: A smaller, role-relevant tool set reduces context overhead, confusion, and cost.
• Control of tool execution: Describe how to control when an agent uses tools and when it should stop using them.
  Why it matters: Explicit controls help prevent unnecessary tool calls.
• Prebuilt tool selection: Explain when to use prebuilt or hosted tools and what to check before consuming an MCP server.
  Why it matters: Reusing tools saves effort only if the tools you adopt are suitable and dependable.
• Tool failure handling: Describe how to prepare an agent for tool failures.
  Why it matters: A failed tool need not leave the agent unable to respond or recover.

Key terms: single-responsibility tools, docstrings, typed params, function tools, schemas, tool bloat, tool sets, MCP servers, tokens, context

11.1.3 Reasoning and planning

This section gives practical ways to guide an agent’s reasoning for complex tasks, bound its work, and avoid unnecessary reasoning for simple tasks.

• Task-appropriate reasoning control: Explain why an agent’s reasoning should be controlled rather than left entirely to a reasoning-capable model, and when reasoning may be unnecessary.
  Why it matters: Matching the amount of reasoning to the task can keep an agent from relying blindly on unpredictable model behavior.
• Complex-task reasoning patterns: Describe how the section recommends guiding an agent through nontrivial tasks and larger goals.
  Why it matters: These patterns give multistep work more structure than an unguided response.
• Iteration and final-review safeguards: Describe the limits and checks the section recommends around an agent’s work.
  Why it matters: A stopping limit and final check help prevent an agent from running indefinitely or leaving obvious errors uncorrected.

Key terms: reasoning, one-shot tasks, ReAct, chain-of-thought, plan-then-execute, sequential thinking servers, iterations, runaway loops, self-review, reflexive pass

11.1.4 Knowledge and memory

This section gives retrieval-first practices for making agent knowledge and memory useful, grounded, scalable, and appropriately accessible.

• Retrieval First Knowledge Design: Explain why retrieval is the starting point for improving agent knowledge and memory, and why a RAG retrieval tool is preferable to putting whole documents in prompts.
  Why it matters: Retrieval lets an agent bring relevant information into context when it needs it.
• Session And Persistent Memory: Distinguish session context from long-term memory, and describe the broader kinds of experience agent memory can retain.
  Why it matters: Separating temporary context from selectively retrieved, persistent memory keeps an agent's context useful over time.
• Scalable And Varied Retrieval: Describe the section's options for scaling retrieval and improving search beyond a single vector-search pattern, including deliberate embedding choices.
  Why it matters: A retrieval design needs to remain useful as the knowledge base grows and search needs vary.
• Coherent Chunks And Grounding: Explain how to prepare retrieved material and constrain an agent's answer when supporting information is absent.
  Why it matters: Coherent evidence and an explicit limit on answers help keep responses grounded in what was retrieved.
• Fresh And Partitioned Knowledge: Describe why a knowledge base needs routine updates and role- or type-based separation, even if a single index initially seems sufficient.
  Why it matters: Retrieval is useful only if its information is current and users receive context they are allowed to access.

Key terms: retrieval, RAG, knowledge base, metadata filters, session threads, long-term memory, vector/DB store, prune, ANN indexes, shard by domain

11.1.5 Evaluation and feedback

This section gives practical ways to measure, evaluate, safeguard, and improve agents after deployment.

• Agent observability and iteration: Explain why agent teams should log and measure both agent behavior and the systems agents rely on, then use those observations to debug and improve performance.
  Why it matters: Observability gives you evidence for fixing incidents and improving an agent rather than guessing.
• Automated and human evaluation: Describe how automated evaluations and captured human feedback help assess and improve agents.
  Why it matters: Automated checks and user feedback reveal whether changes help and where the agent still falls short.
• Guardrails and output safety gates: Describe where guardrails are applied and how an evaluation and safety gate can check agent output.
  Why it matters: Safety checks help catch problematic inputs, tool use, and outputs before they undermine a deployed agent.

Key terms: tracing, observability stack, AIOps, curated test sets, LLM-as-judge, human-in-the-loop (HITL), guardrails, moderation, schema validation, eval and safety gate

11.2 Tips for building a customer support agent

This section applies the five agent layers to customer support, gives practical safeguards, and traces how specialized agents handle a support request.

• Customer support agent layers: Describe how the five agent layers appear in a customer support system: a support persona, lookup and escalation tools, triage reasoning, RAG knowledge, and logs and human feedback.
  Why it matters: The layer mapping provides a checklist for what a support system needs.
• Bounded and grounded answers: Explain why a support agent should start with a limited set of intents and how it should ground answers in provided support material.
  Why it matters: A focused, evidence-based agent is less likely to give an unsupported answer.
• Triage and human escalation: Describe how triage chooses between an answer path and a human handoff, and identify cases that should be escalated.
  Why it matters: Escalation gives the agent a safe path when an automated answer is not appropriate.
• Specialized support workflow: Trace the section’s support workflow from triage through retrieval, grounding, any scoped API action and guardrail check, to the final answer and user feedback.
  Why it matters: Following the handoffs makes clear where evidence, actions, and checks enter the response.
• Identity and scoped access: Explain the access precautions for account data and agent tools in a customer support system.
  Why it matters: Support agents should not reveal account information or take actions beyond the user’s access.
• Reliable and auditable operations: Describe the section’s practical ways to handle failures, protect costs and reliability, and review support decisions.
  Why it matters: Operational safeguards help a support system remain dependable and explain what it did.

Key terms: five agent layers, persona, tools/action, reasoning/planning, knowledge/memory, eval/feedback, narrow the charter, bounded set of intents, RAG, grounding agents

11.3 Tips for building a RAG agent system

The section gives practical guidance for keeping RAG agent systems focused, improving retrieval, grounding answers, evaluating failures, and retrying inadequate searches.

• Selective modular agent roles: Explain when a RAG workflow needs multiple agents rather than a one-shot call, and describe the suggested division of roles.
  Why it matters: A useful RAG workflow does not need extra agents unless they solve a real workflow problem.
• Retrieval quality options: Describe the retrieval improvements the section recommends, including why hybrid retrieval is an option.
  Why it matters: Answer quality depends on the context the system can find.
• Grounded answers and uncertainty: Explain how to keep RAG answers tied to retrieved context and what the system should do when it cannot support an answer.
  Why it matters: Grounding and honest uncertainty help prevent plausible-sounding answers that the retrieved evidence cannot support.
• Retrieval and answer evaluation: Describe what to log or evaluate in a RAG agent system and how retrieval-call tracking helps.
  Why it matters: Observing search and answer failures shows where the workflow needs improvement.
• Corrective retrieval loop: Trace the section’s example of checking retrieved documents, refining an inadequate query, and preventing endless retries.
  Why it matters: A corrective pass can improve inadequate search results without letting the workflow retry forever.

Key terms: one-shot RAG call, orchestrator, triage (router), retriever, answerer, critic, ANN indexes, metadata filters, domain-tuned embeddings, reranking

11.4 Tips for building a deep research agent system

This section gives practical guidance for deciding when to build an internal deep research agent and for organizing its research, fact-checking, reporting, and operational controls.

• Internal research agent fit: Explain what deep research agents do and when building an internal version is warranted.
  Why it matters: The extra complexity should serve a research need that a simpler system cannot meet.
• Planner and worker roles: Describe the two-tier arrangement of a research planner and specialized workers, including when a worker-like tool is itself an agent.
  Why it matters: Separating coordination from specialized work makes the research pattern easier to organize.
• Source-backed research policy: Explain how tool-use rules and citations keep factual claims tied to retrieved sources.
  Why it matters: Research is more useful when readers can connect its claims to the material the agent found.
• Critic-guided research cycle: Trace how the critic checks retrieved context and either requests more research or passes it to a writer for a final answer.
  Why it matters: The critic provides a checkpoint before research findings become a final answer.
• Streaming research progress: Describe what to stream during research and why the section recommends it.
  Why it matters: Visible progress helps users follow and approve a longer-running research process.
• Caching and run budgets: Explain the recommended controls on repeated queries and resource use, including the downside of stale cached results.
  Why it matters: Resource controls help repeated research runs without letting old search results undermine them.

Key terms: deep research agents, internal version, two-tier orchestration, research planner, stateless workers, tools, agents that use tools, tool policy controls, retrieval and web tools, references

11.5 Summary

Summarizes how to assemble production-ready agentic systems from five design layers, then applies those principles to customer support, RAG, and deep research.