liveBook · Manning

Chapter 8: Human In the Loop

What you will learn in chapter 8 of Build a Multi-Agent System (from Scratch)

This introduction explains when human input, result approval, and stepwise supervision can help with agent tasks that should not be fully autonomous.

After this chapter you will be able to

What it builds on

Section by section

Introduction

This introduction explains when human input, result approval, and stepwise supervision can help with agent tasks that should not be fully autonomous.

Key terms: tool-based HITL, human input, result-approval HITL, human approval gate, task results, supervised HITL, stepwise task execution, processing loop

8.1 When—and How—should humans be in the loop?

This section explains when human involvement can resolve ambiguity, reduce high-stakes errors, or address security and compliance concerns in agent execution.

Key terms: ambiguous task execution, human-input tool, Tool-Based HITL, high-stakes task execution, result-approval step, Result-Approval HITL, supervised execution, Supervised-Execution HITL, result approval, security risks

8.2 Pattern 1: Tool-Based HITL

This section implements HumanInputTool so an agent can request human input during task execution without changing its processing loop.

Key terms: Tool-Based HITL, HumanInputTool, processing loop, BaseTool, parameters_json_schema, prompt, choices, __call__(), validate_tool_call_arguments(), _prompt_human()

8.3 Pattern 2: Result-Approval HITL

This section explains how human review determines whether an agent’s task result is accepted or returned for further processing.

Key terms: Result-Approval HITL, task result, human feedback, ApprovalResult, RejectedTaskResult

8.3.1 Updating TaskHandler and LLMAgent

This section updates LLMAgent and TaskHandler to review task results with a human and resume execution when a result is rejected.

Key terms: with_approval, TaskResult, ApprovalResult, RejectedTaskResult, request_approval(), get_next_step(), TaskStepResult, TaskStep, run_step(), _prompt_for_approval()

8.4 Pattern 3: Supervised-Execution HITL

This section explains how supervised execution gives a human control over an agent’s processing loop and outlines the handler that supports it.

Key terms: Supervised-Execution HITL, processing loop, get_next_step(), run_step(), LLMAgent.run_supervised(), SupervisedTaskHandler, TaskHandler, episodic memory

8.4.1 Implementing SupervisedTaskHandler

This section describes how SupervisedTaskHandler supports human review while reusing TaskHandler capabilities and preventing automatic execution.

Key terms: SupervisedTaskHandler, TaskResult, complete(), reject(), RejectedTaskResult, Result-Approval HITL, TaskHandler, get_next_step(), run_step(), rollout

8.4.2 Updating LLMAgent

This section adds supervised execution to LLMAgent and demonstrates how a human advances, inspects, and finishes a task.

Key terms: run_supervised(), run(), max_steps, with_approval, SupervisedTaskHandler, memories, get_next_step(), TaskStep, run_step(), TaskStepResult

8.4.3 Introducing the LLM Agent Inspector for supervised task executions

This section shows how to set up and use the LLM Agent Inspector to supervise and review an agent’s task execution through a browser UI.

Key terms: LLM Agent Inspector, agent-inspector, LLMAgentBuilder, agent_builder, default_task, Create session, get_next_step(), run_step(), skill discovery, scope

8.5 Extending to MAS: HITL

This section assesses how three human-in-the-loop patterns extend from individual LLM agents to multi-agent systems.

Key terms: HumanInputTool, Tool-Based HITL, synchronization, locking, Result-Approval HITL, final aggregated task result, human oversight, Supervised-Execution HITL, hierarchical MAS, coordinator LLM agent

8.6 Summary

Where this is used later

Concepts explained in this chapter

← Chapter 7: Memory Chapter 9: Assembling MAS with Subagents →

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.