liveBook · Manning

Chapter 3: Multi-agent pipelines and orchestration

What you will learn in chapter 3 of AI Agents for Offensive Security

This chapter introduces multi-agent offensive-security pipelines in which specialized agents exchange artifacts under orchestration, safety controls, and observable failure handling.

After this chapter you will be able to

What it builds on

Section by section

Introduction

This chapter introduces multi-agent offensive-security pipelines in which specialized agents exchange artifacts under orchestration, safety controls, and observable failure handling.

Key terms: multi-agent pipelines, single-agent designs, specialized agents, staged workflow, artifacts, agent interfaces, loose coupling, observable, record, orchestration

3.1 Why multi-agent systems?

This section explains why multi-agent pipelines are needed to coordinate specialized agents across the dependent stages of real offensive-security workflows.

Key terms: single agents, ReAct loop, multi-agent systems, reconnaissance, raw observations, prioritization, evidence, artifacts, multi-agent pipelines, orchestration

3.2 Multi-agent mental model

This section establishes a mental model for multi-agent systems as focused single agents connected by artifact-based pipelines in different routing topologies.

Key terms: multi-agent systems, single agents, reasoning pipeline, monolithic agent, staged pipeline, artifacts, outputs, inputs, persisted, physical files

3.3 Artifacts as agent interfaces

This section establishes artifacts as the exclusive, contract-like interfaces through which agents exchange context, decisions, and provenance in a multi-agent pipeline.

Key terms: artifacts, agent interfaces, downstream agents, in-memory state, metadata, traceability, context, log file, decision, resumption

3.4 Artifact provenance, replay, and auditability

This section explains how disciplined, durable artifacts make multi-agent pipelines pausable, replayable, auditable, and transparent to human reviewers.

Key terms: artifacts, persistent outputs, triage, reconnaissance, report, human in the loop, replay, known artifact, verifiable execution trail, auditability

3.5 Agent orchestration and execution control

This section introduces orchestrators as the guiding force that organizes agents in a multi-agent system toward shared goals.

Key terms: orchestrators, multi-agent systems, execution control

3.5.1 What the orchestrator does

This section explains how an orchestrator coordinates multiple agents by explicitly controlling task routing, state, data flow, and execution order.

Key terms: orchestrator, task decomposition and routing, state management, agent coordination, decision logic, workflow state, reconnaissance, triage, prioritization, exploitation planning

3.5.2 3.6.2 Why do we need an orchestrator?

This subsection justifies a centralized orchestrator by showing how it streamlines decisions and communication while leaving specialized agents focused on their own roles.

Key terms: orchestrator, centralized design architecture, streamlined decision-making, communication pathways, worker agents, specialized agents, reasoning and action, specific roles

3.6 Shared state and memory boundaries

This section explains how deliberately structured, artifact-based shared memory preserves state across agents and time without making agents inherently smarter.

Key terms: agentic memory, shared state, execution window, artifacts, memory layers, authoritative state, auditability, explicit fields, timestamps, source identifiers

3.7 Safety gates and authorization control

This section introduces why safety gates and authorization control are necessary when offensive-security workflows are expanded from single agents to multi-agent systems.

Key terms: dual-use, authorization boundaries, operational disruption, legal exposure, safety gates, multi-agent systems, authorization control, single agents

3.7.1 Applying safety gates in multi-agent systems

This section explains why every multi-agent step needs an external safety gate rather than allowing an LLM agent to judge its own authorization or safety.

Key terms: safety gates, agentic step, approval layers, automated pipeline, policy enforcement, probabilistic system, non-deterministic, authorization, guardrails, safety mechanisms

3.7.2 Human in the loop

This subsection explains how human review at safety gates keeps multi-agent security pipelines within authorized, legal, and risk-appropriate boundaries while producing auditable approval evidence.

Key terms: human in the loop, safety gates, engagement scope, client-specific risk tolerance, legal constraints, operational judgment, formalize, review, approval, defensible evidence

3.8 Error handling and resilience

This section introduces resilience as the ability of a production agent pipeline and its orchestrator to handle failures gracefully rather than crash.

Key terms: pre-defined schemas, logging, LLM API calls, unexpected format, robustness, orchestrator, recover gracefully, plan B, graceful exits, resilience

3.8.1 Retry logic and exponential backoff

This section explains how retry logic with exponential backoff lets agent pipelines survive transient external-service failures while surfacing non-transient errors immediately.

Key terms: transient errors, retry logic, agentic pipeline, exponential backoff, retry attempt, delay interval, exceptions, syntax errors, authentication failures

3.8.2 Checkpointing

This section explains how checkpoints save stage artifacts so an interrupted pipeline can resume without repeating completed work.

Key terms: checkpoints, save points, orchestrator, artifacts, pipeline stages, JSONL file, stage name, artifact, load into memory, newest result artifact

3.8.3 Error classes and mitigation strategies

This subsection explains that an orchestrator should classify failures and apply a response appropriate to each error rather than treating all errors alike.

Key terms: error classes, mitigation strategies, orchestrator, retry, skip, pause

3.8.4 Monitoring, metrics, and alerts

This section explains how to monitor multi-agent systems with operational, cost, quality, and performance metrics, then alert, prioritize, and retain telemetry for response and analysis.

Key terms: error rates, error type, token usage, cost per agent, gold standard, hallucination rates, lightweight hooks, pipeline health, error-count hook, alert

3.8.5 Building confidence in AI agents through resilience

This subsection concludes that resilient orchestrator handling turns probabilistic multi-agent automation into a safe, explainable, repeatable workflow and leads into visibility through traces and audits.

Key terms: orchestrator, controlled scientific workflow, repeatable, red teams, blue teams, probabilistic machines, cascade, catastrophic failure modes, multi-agent system

3.9 Metric visualization and auditing AI agents

This section introduces visualization and auditing as the way to make a multi-agent pipeline’s end-to-end execution visible and accountable.

Key terms: visual trace, well-instrumented pipeline, visualize an agent’s process, compliance, accountability, rapid debugging, anomalies, auditors, trustworthy system

3.9.1 Generating simple traces

This subsection defines traces and spans and explains how existing artifact logs and timestamps can be summarized to visualize an agent run.

Key terms: trace, span, function, operation, start timestamp, end timestamp, input/output data, metadata, end-to-end visibility, agent chains

3.9.2 Graph-based views

This section shows how to turn pipeline stage lists and artifact metadata into graph-based visualizations, especially Mermaid flowcharts, to make control flow easier to interpret and document.

Key terms: stage list, artifact metadata, Mermaid diagram, Graphviz diagram, control flow, Mermaid syntax, Markdown, version control systems, GitHub, live pipeline diagrams

3.9.3 Correlating artifacts and logs

This subsection explains how correlating logs with persisted artifacts and visual views makes multi-agent failures easier to diagnose and audit retrospectively.

Key terms: logs, artifact data, schematics, graphs, root causes, multi-agent systems, persistent storage, replay, audit trail, audit chain

3.9.4 Defense through transparency

This subsection explains how transparent red-team pipelines support defensive AI improvement, blue-team operations, and audit evidence.

Key terms: transparent red-team pipeline, security-specific AI models, detection logic, anomaly detection, incident response readiness, blue team, monitoring and detection, incident response, defense implementation, threat intelligence

3.9.5 Practical visualization tips

This section recommends keeping pipeline visualizations current, choosing visualization tools by operational need, and using transparency to make agent activity explainable across teams.

Key terms: artifacts, trace.txt, graph.mmd, JSONL logs, total runtime, gate outcomes, Grafana, Streamlit, Jupyter notebooks, real-time dashboards

3.10 Reconnaissance multi-agent pipeline

This section introduces the reconnaissance multi-agent pipeline as a coherent integration of agents, artifacts, orchestration, safety gates, and shared state.

Key terms: multi-agent system, pipeline, agents, artifacts, orchestration, safety gates

3.10.1 Pipeline overview

Introduces a three-agent reconnaissance pipeline whose externally managed artifacts, safety gates, shared state, and orchestrator separate coordination concerns from agent responsibilities.

Key terms: ReconNormalizeAgent, TriageAgent, ReportAgent, raw recon data, findings, rationale, external orchestrator, execution order, shared state, safety gates

3.10.2 ReconNormalizeAgent

This subsection defines ReconNormalizeAgent as the pipeline’s narrow input-normalization stage and places schema validation and scope enforcement at its boundary.

Key terms: ReconNormalizeAgent, raw recon output, predictable artifact schema, normalization, host, path, status, title, derived signals, timestamp

3.10.3 TriageAgent

This subsection defines the TriageAgent as a non-interactive prioritization step that produces a reviewable triage artifact before human approval or denial.

Key terms: TriageAgent, normalized artifact, prioritization logic, live interaction, triage artifact, priority, rationale, evidence references, timestamp, human-in-the-loop gate

3.10.4 ReportAgent

This subsection introduces ReportAgent as the final pipeline stage that turns a triage artifact into human-consumable output while marking suggested next steps for approval.

Key terms: ReportAgent, triage artifact, human consumption, suggested next steps, requiring approval, HTTP titles, login pages, orchestrator, Mermaid graph, artifact sizes

3.11 Failure modes in multi-agent systems

This introduction frames failures in multi-agent pipelines as expected operational realities and defines robustness by making them visible, bounded, and recoverable.

Key terms: fragile system, robust system, visible, bounded, recoverable, failure modes, multi-agent pipelines, design lessons, disciplined architecture

3.11.1 Silent artifact drift

This subsection explains how unnoticed changes to artifact structure or meaning can silently corrupt a multi-agent pipeline and how contract-based artifact handling exposes such failures.

Key terms: silent artifact drift, artifacts, downstream agents, cascading errors, artifacts as contracts, schema versioning, explicit validation, schema metadata, fail loudly

3.11.2 Orchestration hidden inside agents

This subsection explains why execution control must stay in an external orchestrator rather than being embedded in individual agents.

Key terms: orchestration, orchestrator, control flow, execution order, deterministic, orchestration logic, conditional, retry, implicit execution order, fragmented control logic

3.11.3 Safety gates treated as formalities

This subsection explains why safety gates fail when teams treat them as bypassable formalities and what makes a gate meaningful in a workflow.

Key terms: safety gates, rubber stamps, workflow failure, decision-relevant information, judgment

3.11.4 Memory accumulation without correction

This section explains why accumulated agent memory can cause pipelines to drift and why memory must remain correctable, artifact-derived, and non-authoritative.

Key terms: memory, artifacts, derived, authoritative, summaries, replay, memory accumulation, drift, recon data, stale

3.11.5 Overloaded agents

This section identifies overloaded agents as a multi-agent failure mode and argues for splitting responsibilities into separate pipeline stages.

Key terms: overloaded agents, artifacts, pipeline, responsibility, split the pipeline, complexity

3.12 Summary

This summary ties specialized agents, artifacts, orchestration, safety, resilience, and visualization into a repeatable multi-agent red-team workflow.

Where this is used later

Concepts explained in this chapter

← Chapter 2: Building your first AI agent Chapter 4: Passive reconnaissance agents →

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.