Chapter 4: Architecting and building multi-agent systems

Introduction

The chapter introduction explains why multi-agent design shifted from simply adding agents toward structuring their roles, while noting the tradeoff between single-agent simplicity and capability.

• Structured multi-agent design: Explain why early enthusiasm for adding more agents gave way to structured multi-agent architectures, using the contrast between unconstrained subgoal loops and teams with defined roles and communication channels.
  Why it matters: The chapter’s architectures respond to a practical failure of simply adding more agents.
• Single-agent coordination tradeoff: Distinguish the coordination advantage of a lone agent from the limitations the introduction identifies for complex tasks.
  Why it matters: This tradeoff helps explain why a more complex multi-agent workflow may be worth considering.

Key terms: unconstrained loops, role-based teams, defined responsibilities, communication channels, single-agent systems, multi-agent coordination, tool use, reasoning and planning

4.1 Architecting multi-agent systems

This section introduces three multi-agent architecture patterns and shows how decision-making, control, and communication distinguish them.

• Decision and execution authority: Distinguish decision-making—the authority to choose the next action using available context—from control—the authority to execute it. Explain that different agents can hold these authorities and that a decision-maker needs appropriately scoped context.
  Why it matters: Separating the choice of an action from its execution makes an architecture’s responsibilities clearer.
• Agent communication boundaries: Explain that communication concerns which context passes between agents and how. Contrast shared context in collaboration with selective communication through an orchestrator, including the trade-off between coordination and context bloat or redundant work.
  Why it matters: Who sees whose work affects both coordination and efficiency.
• Three architecture patterns: Describe how flow, orchestrator, and collaboration patterns arrange decision-making, control, and communication differently: an agent-to-agent assembly line, a central decision-maker delegating execution, and agents passing decisions and control among themselves through a shared channel.
  Why it matters: These distinctions let you describe an architecture by how its agents actually work together, rather than by its name alone.
• Deliberate architecture choice: Justify examining how decision-making, control, and communication will move before choosing a multi-agent pattern. State that patterns can be mixed, no pattern is mandatory for a use case, and additional agents can bring cost, latency, and unpredictability despite their greater potential.
  Why it matters: A more elaborate multi-agent design is not automatically the right design for a task.

Key terms: decision-making, control, context, communication, common context, orchestrator, context bloat, flow pattern, orchestrator pattern, collaboration pattern

4.1.1 Decision-making and control patterns

This section compares where decisions and control reside in three multi-agent patterns and offers a rule of thumb for choosing among them.

• Three decision and control patterns: Distinguish agent flow, orchestration, and manager-worker hierarchy by who makes decisions, delegates work, and controls task execution.
  Why it matters: The patterns differ chiefly in where decisions and control sit, which shapes how agents work together.
• Control and complexity tradeoff: Explain the section's rule of thumb for choosing a pattern when greater control or delegation is difficult to manage.
  Why it matters: Extra structure is useful only when you can manage the coordination it requires.

Key terms: agent flow, agent orchestration, manager-worker pattern, decision-making, control, DM/Control/Communication, delegation patterns, flow pattern

4.1.2 Communicating with shared memory, message passing, and MCP

The section explains why multi-agent systems should limit shared context and compares ways agents can communicate.

• Costs of shared context: Explain why giving agents more shared context than they need increases cost and can reduce their ability to select relevant information.
  Why it matters: Limiting context can improve both efficiency and the quality of an agent’s response.
• Task specific communication boundaries: Describe what the research and synthesis subagents receive in the competitive-analysis example and why their inputs differ.
  Why it matters: The example shows how to make communication boundaries deliberate rather than sharing everything by default.
• Agent communication patterns: Distinguish sequential message passing, shared conversation access, and restricted communication through an agent used as a tool, and describe the section’s advice on combining patterns.
  Why it matters: The communication pattern determines which agents can see information at each step.

Key terms: shared context, selection accuracy, context dilution, lost in the middle, research orchestrator, research subagents, synthesis subagent, communication boundary, rubric, output schema

4.1.3 Channeling multi-agent coordination strategies

This section explains how to choose and combine multi-agent coordination patterns while balancing agent autonomy against the control a reliable system needs.

• Agency and constraint balance: Explain why agents need room to decide within a system that controls consequential boundaries, and how the amount of freedom should change with risk and demonstrated reliability.
  Why it matters: This balance helps you give agents useful discretion without surrendering control where errors matter most.
• Sequential and parallel execution: Distinguish a sequential pipeline from parallel delegation, including how work and outputs move, when each fits, and the principal costs of choosing a pipeline.
  Why it matters: Choosing between dependent stages and concurrent independent work affects speed, failure handling, and workflow complexity.
• Hierarchical orchestrator responsibilities: Distinguish hierarchical coordination from parallel flow by what the orchestrator actually does, and identify when that added management is worthwhile.
  Why it matters: It prevents you from adding a planning agent when simple parallel delegation would suffice.
• Iterative critique and debate: Explain how repeated proposal, critique, and revision improve an output; distinguish peer debate from critic-gated refinement and describe how either loop can stop.
  Why it matters: Iteration can improve ambiguous or high-stakes work, but its roles and exit rule determine whether it is controlled and worthwhile.
• Ensemble voting and diversity: Describe how voting or best-of-N selects among multiple candidate answers, and explain why useful diversity matters more than simply increasing the number of agents.
  Why it matters: Multiple answers improve reliability only when the candidates are different enough to avoid repeating the same mistake.
• Other collaboration and routing shapes: Distinguish role-playing collaboration, conditional routing, and peer-to-peer networks by how agents interact or receive work, and state a key situation or limitation for each.
  Why it matters: These shapes solve different coordination needs, so recognizing their trade-offs helps avoid unnecessary conversation, routing, or decentralization.

Key terms: agency, constraint, flow pattern, human-in-the-loop checkpoints, sequential pipeline, agent-to-agent pipeline, parallel flow, output schema, retry-with-backoff, hierarchical coordination

4.2 Balancing agents with agentic flows

Introduces agent flows as a way to make an agent’s actions easier to break down and understand, even when a single agent is otherwise preferred.

• Value of agent flows: Explain why a flow of agents can be useful instead of a single monolithic agent.
  Why it matters: This gives you a reason to consider an agent flow without assuming that more agents are always better.

Key terms: single agent, monolithic agent, agent flow

4.2.1 Transforming agents to agent flows

This section explains when and why to turn a single agent into a flow of specialized agents, focusing on overload, tool selection, and per-call cost.

• Role Based Agent Decomposition: Explain why a single agent can suit simple tasks but may need to become an agent-to-agent flow as its responsibilities grow. Describe the resulting agents as having distinct, specialized roles.
  Why it matters: It helps you recognize when adding more to one agent is less useful than giving agents separate jobs.
• Overload And Tool Confusion: Describe how too many tools or a long prompt can make an agent fail at tasks, including choosing the wrong tool when descriptions overlap. Explain the stated remedy for overlapping tool descriptions.
  Why it matters: It separates a reason to reduce an agent's load from a specific fix for ambiguous tool choices.
• Repeated Tool Context Costs: Explain why restricting specialized agents to relevant tools and communication can reduce cost and latency. State why a large tool set adds overhead on every LLM call.
  Why it matters: Focusing each agent's available context can save tokens repeatedly, not just at setup time.

Key terms: single agent, agent-to-agent flow, specialized roles, responsibilities, overload, attention overload, tool descriptions, selection accuracy, cost/latency, context

4.2.2 Building an agent-to-agent flow

This section explains how to turn a single agent into an agent-to-agent flow by assigning specialized roles and tools to separate agents.

• Agent flows and prompt chains: Distinguish an agent-to-agent flow from prompt chaining: both pass output to a next step, but agents can also make decisions through tool use.
  Why it matters: The comparison clarifies what changes when chained steps are agents rather than prompts.
• Role-based agent decomposition: Explain how to split a single agent by the roles it performs, specializing agents and assigning each the instructions and tools appropriate to its role. Describe the section’s example of creating an agent for each MCP server.
  Why it matters: Role-based splitting gives each agent a clearer responsibility and a tool set suited to it.
• Benefits of agent flows: Describe what converting a single agent into a flow makes possible: multiple agent steps, MCP tools, other output forms, and some control over decision-making for greater predictability.
  Why it matters: These are the practical reasons the section gives for turning one agent into a flow.

Key terms: agent-to-agent flows, prompt chaining, tool use, roles, specializing agents, ownership, MCP server, agent flow, MCP tools, deterministic

4.2.3 Agency and decision-making in agent flows

This section shows how to make an agent-to-agent flow more predictable by moving a decision into code, constraining research output, and passing focused inputs between agents.

• External Decision Point: Explain why a hard decision rule can be moved out of an LLM-powered agent and into code in an agent flow.
  Why it matters: Code can enforce a rule that an agent might apply inconsistently.
• Structured Research Sources: Describe how a constrained list of strings makes the research agent’s output easier to check and use.
  Why it matters: A predictable output shape lets the flow make a straightforward check even when the number of sources varies.
• Research Flow Branches: Trace what the example flow does when the research agent returns no sources versus one or more sources.
  Why it matters: Both branches give the downstream filesystem agent something to receive despite variable research results.
• Focused Interagent Inputs: Explain why the example sends labeled, limited inputs to an agent rather than passing every step’s conversation history.
  Why it matters: Focused inputs help agents use the relevant information without carrying an entire conversation through the flow.

Key terms: agency, deterministic, decision point, guardrail, ResearchSourcesModel, structured output, constrained list of strings, search sources, static string, thinking agent

4.3 Understanding handoffs in agent flows

This section compares three agent-flow communication patterns by how they share context, control interactions, and handle overhead or failures.

• Conversational flow trade-off: Explain how conversational flow puts agents on one conversation thread and why shared context also creates overhead.
  Why it matters: A shared thread makes context available to every agent, but each agent carries the extra conversation content.
• Pass-off code control: Describe how external code controls each pass-off between agents and explain the trade-off between precise control and manual implementation.
  Why it matters: Pass-offs offer precise control, but the calling code must manage the interactions and their data carefully.
• Guardrailed handoff trade-off: Explain the stated advantage and disadvantage of the enhanced agent handoff with a guardrail.
  Why it matters: This pattern offers a way to retain context with less token overhead, while introducing a recovery concern when handoffs fail.

Key terms: conversational flow, conversation thread, context, overhead, pass-off communication system, typing, formatting, code complexity, enhanced agent handoff, guardrail

4.3.1 Agent-to-agent flow with handoffs

Explains how agent-to-agent handoffs transfer control without coded transitions, while making agents’ connections less explicit.

• Agent-to-agent handoff requirements: Explain how a handoff transfers control internally without a separately coded transition, and why an agent’s instructions must name the agent receiving the handoff.
  Why it matters: It explains both what handoffs replace and what agents must specify for the flow to work.
• Handoff simplicity and visibility tradeoff: Describe why handoffs simplify orchestration code and make transitions more seamless, while making flow dependencies harder to inspect.
  Why it matters: Simpler code does not necessarily make a complex agent flow easier to understand.

Key terms: handoff pattern, agent instructions, orchestration code, dependencies, agentic system

4.3.2 Visualizing agent flows

This section shows how to visualize agent handoffs and inspect their traces, then explains why agent naming matters as systems grow.

• Flow graphs and traces: Distinguish the automatically rendered agent-to-agent flow graph from the OpenAI Dashboard Traces page. Explain what each view helps you see and what the graph does not yet show.
  Why it matters: The overview and the detailed calls answer different questions about an agent flow.
• Agent names at scale: Explain why agent names must be managed consistently and how a naming convention or agent registry helps as the number of agents grows.
  Why it matters: Stable identifiers and a central lookup keep a growing agent system easier to navigate and change.

Key terms: handoff pattern, flow graph, draw_graph, Traces page, MCP servers, agent names, handoff routing, naming convention, agent registry, hardcoded agent imports

4.3.3 Monitoring the handoff

This section explains how to observe data at an agent handoff and why guardrails, rather than callbacks, should handle bad data.

• Handoff data visibility: Explain why handoffs need monitoring and how a handoff wrapper callback makes transferred data and the reason for a handoff visible.
  Why it matters: Seeing what crosses a handoff helps you investigate agent behavior that is not performing as expected.
• Callbacks versus guardrails: Distinguish using callbacks to observe and log handoff data from using guardrails to block or correct bad data.
  Why it matters: This distinction keeps handoff monitoring separate from enforcing data checks.

Key terms: agent handoff, handoff wrapper function, callback function, callbacks, guardrails, handoff point

4.4 Validating agent flows with guardrails

This section explains why agent flows need guardrails and how to choose and layer validation methods according to risk, cost, and the kind of judgment required.

• Guardrails in agent flows: Explain where agent flows need validation and how a guardrail differs from filtering output after the fact.
  Why it matters: The point of a guardrail is to catch a problem before the flow commits to a risky action.
• Validation methods and costs: Distinguish the risks suited to deterministic checks, classifier models, and LLM-based guardrails, and explain when the extra LLM cost is justified.
  Why it matters: Matching the check to the risk avoids paying for an LLM judgment when a cheaper check will do.
• Layered guardrail defense: Describe the section’s defense-in-depth approach and why an LLM guardrail should not be the only check.
  Why it matters: Layering checks limits cost and avoids relying on a validator that can share the agent’s mistakes.

Key terms: guardrails, agent flows, agent-to-agent transfers, semantic circuit breaker, LLM-based guardrails, regex, schema validation, classifier models, code-based validation, semantic

4.4.1 Implementing input and output guardrails

This section explains why agent systems need input and output guardrails and shows how simple checks can flag invalid data.

• Guardrail purpose and boundaries: Explain why variable inputs and outputs make guardrails useful in AI agent systems, and identify the data they can control.
  Why it matters: This explains where checks can help protect an agent flow.
• Input and output validation: Describe how the example guardrails check a single agent’s input and output and what happens when a check fails.
  Why it matters: The example makes guardrails concrete as checks around an agent rather than just a general safety idea.

Key terms: guardrails, input, output, agent to agent, input guardrail, output guardrail, tripwire flag, string comparisons, length checks

4.4.2 Using agents as guardrails

This section shows how an agent can replace simple input and output checks as a guardrail, while noting the added complexity and the Agents SDK’s handoff limitation.

• Agent-powered validation tradeoffs: Explain how a guardrail agent can replace string matching and length checks for input and output, and why prompt-based validation is both flexible and risky.
  Why it matters: It explains why you might choose an agent as a guardrail instead of a simple check, without overlooking the risk.
• SDK guardrails and handoffs: Describe what Agents SDK guardrails control and distinguish their intended use from guardrails in agent-to-agent communication.
  Why it matters: The SDK’s handoff limitation does not rule out guarding communication between agents.

Key terms: guardrail agent, validation rules, string matching, length checks, Agents SDK, guardrails, handoffs, agent-to-agent communication

4.4.3 Adding guardrails for pass-off agent flows

This section explains how to protect agent handoffs with guardrails and bounded retries, then weighs orchestration against simpler agent flows.

• Handoff failure modes: Distinguish shape failure, context leakage, and instruction injection in agent handoffs, and explain why instruction injection is especially dangerous.
  Why it matters: Recognizing the distinct risks helps you decide what a handoff must protect against.
• Guardrails at critical boundaries: Explain where to place handoff guardrails and how the section’s pass-off example handles an insufficiently detailed plan.
  Why it matters: Checking consequential handoffs can prevent a bad pass-off without burdening every exchange.
• Bounded recovery and retries: Describe how retrying the thinking agent can recover from an insufficient plan, why retries must be bounded, and how transient-error retries should be spaced.
  Why it matters: A recovery loop is useful only if it has a safe stopping point and does not amplify temporary failures.
• Orchestration control and tradeoffs: Explain how converting a flow node to an orchestrator changes delegation, and describe the limitation observed in this example.
  Why it matters: More direct control over agents does not automatically produce a better or more reliable result.
• Simple flow improvements: State the section’s preferred ways to strengthen an agent flow before adding architectural complexity.
  Why it matters: Small, targeted changes may make a flow more robust without a harder-to-manage design.

Key terms: shape failure, context leakage, instruction injection, guardrails, high-stakes handoffs, pass-off agent flow, filesystem agent, retry mechanism, recovery and retry loops, exponential backoff with jitter

4.5 Exercises

Five exercises guide the reader through building a two-agent flow, adding a typed decision, switching to SDK handoffs, inspecting the agent graph, and protecting plans with an output guardrail and retries.

4.6 Summary

The section summarizes how to structure multi-agent flows and make their coordination, handoffs, and operation reliable.