liveBook · Manning

Result-approval hitl

As explained in Build a Multi-Agent System (from Scratch)

Result-Approval HITL is a human-in-the-loop pattern in which a human reviews and either approves or rejects an LLM agent's completed task result.

A sensitive memo destined for every employee can benefit from a final human review before it is accepted. Result-Approval HITL places a human approval gate after an LLM agent produces a task result. Approval closes the loop; rejection sends the result and written feedback back for further processing. section 8.0 section 8.3

The gate is enabled by the caller and enforced by the framework, rather than requested at the agent’s discretion. It is lighter than supervised execution, which gives the operator control over individual steps. Result approval catches problems at the end, but cannot prevent errors from affecting earlier steps. section 8.1 section 8.3.1

A processing loop can implement the gate with these changes:

  • Add with_approval to LLMAgent.run(); when enabled, request review after the TaskHandler produces a TaskResult. section 8.3.1
  • Use TaskHandler.request_approval() to present the result and return an ApprovalResult containing the operator’s decision and feedback. section 8.3 section 8.3.1
  • On approval, record the episode in memory and set the handler’s result, marking it done. section 8.3.1
  • On rejection, wrap the failed result and feedback in a RejectedTaskResult and pass it to the next get_next_step() call. The rejection path creates a corrective task step rather than relying on LLM routing. section 8.3 section 8.3.1

The example prompt uses Rich to display the proposed result, ask for confirmation, and collect correction rationale on rejection. request_approval() runs the blocking prompt in a thread via asyncio.to_thread(). Its EOF handler auto-approves in a headless environment, so enabling the flag alone does not guarantee a human review in that case. section 8.3.1

Revising a sequence result

An agent computes the Hailstone sequence from 5 with with_approval=True. The operator rejects the proposed result and asks to omit its first sentence. The loop generates a corrective step, presents the revised result, and completes after approval. section 8.3.1

Figure 8.9 The sequence of interactions between a human operator, LLMAgent, and the TaskHandler it dispatches for carrying out the Result-Approval HITL pattern

Follow the interaction between the agent, task handler, and operator through approval and rejection. section 8.3.1

In a multi-agent system, the approval gate can apply to individual agent tasks or to the final aggregated result. Choose the review point partly by considering whether errors need earlier oversight to avoid costly downstream effects. section 8.5

How Build a Multi-Agent System (from Scratch) covers it

How deep it goes
Walks through approval data structures, asynchronous prompting, loop updates, and rejection routing, while deferring the data structures’ implementations.
Examples
Uses a sensitive employee memo and a Hailstone sequence revision to illustrate review and feedback.
What you can do after
Builds an approval gate into an agent processing loop with feedback-driven revision.
How it connects
Connects human-in-the-loop approval to agent processing loops, supervised execution, and multi-agent systems.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Build a Multi-Agent System (from Scratch). The section links point to the parts of the book it draws on.