chapter eight

8 Human In the Loop

 

This chapter covers

  • When should humans be in the loop to assist or oversee the execution of tasks?
  • Three different patterns for human-in-the-loop task executions
  • Implementing the human input tool, result approval gate, and supervised task execution in our framework

We’ve come a long way since implementing our first LLM agent. Adding support for MCP tools, Skills, and (episodic) Memory, has enhanced it by enabling it to accomplish tasks more effectively and efficiently. But there are some tasks that an LLM agent cannot, or perhaps should not, perform autonomously. That’s why we complete our stage 2 build by supporting three different approaches to enabling human-in-the-loop (HITL) task execution.

The first of these three approaches will have us define a tool, called HumanInputTool, that LLM agents can use at their discretion to receive input from a human operator at any point during task execution. Tool-based HITL is useful when task execution becomes ambiguous and human input is required to continue making progress, such as when an LLM agent is tasked with finding a restaurant in an area of a city but can’t narrow the options without knowing the types of cuisine the human prefers.

8.1 When—and How—should humans be in the loop?

8.2 Pattern 1: Tool-Based HITL

8.3 Pattern 2: Result-Approval HITL

8.3.1 Updating TaskHandler and LLMAgent

8.4 Pattern 3: Supervised-Execution HITL

8.4.1 Implementing SupervisedTaskHandler

8.4.2 Updating LLMAgent

8.4.3 Introducing the LLM Agent Inspector for supervised task executions

8.5 Extending to MAS: HITL

8.6 Summary