Human-in-the-loop (HITL) is an oversight pattern that requires human review before an AI system executes high-impact actions. It combines automated throughput with human judgment, allowing reviewers to validate outputs, resolve ambiguity, authorize actions, and add context the model lacks. section 9.1.6 section 9.2
HITL differs from human-on-the-loop monitoring, where execution is autonomous but a person can intervene in real time. Human-out-of-the-loop execution is suited to low-risk, reversible tasks and relies on post-hoc auditing. section 9.1.6
Table 9.2 Risk-based selection of human oversight controls
| Condition | Oversight control | Operational rule |
Low-risk, reversible response |
Automated response with sampling and audit |
The assistant answers directly, while a sample of interactions is reviewed for quality. |
Uncertain or sensitive interaction |
HOTL monitoring or escalation trigger |
The assistant may continue, but confidence, sentiment, or policy signals can route the case to a human. |
High-impact or irreversible action |
HITL approval |
The assistant drafts the action, but execution requires human approval. |
Legal, safety, regulatory, or complaint case |
Human-owned workflow |
The assistant stops autonomous handling and transfers the case to a qualified reviewer. |
Use risk, sensitivity, and reversibility to select an oversight control, from sampled automated responses to human-owned workflows. section 9.1.6
A production HITL pipeline makes the review boundary explicit:
- Generate a proposed response or action, then evaluate it against confidence and risk thresholds. section 9.2
- Execute low-risk operations automatically; route high-risk or low-confidence cases to a review queue. section 9.2
- Let an operator inspect, edit, or approve the proposal before execution. Record the decision and modifications in a trace store. section 9.2
- Feed verified interactions into evaluation datasets and benchmark suites. section 9.2
Customer support escalation
A support assistant can route complaints, refund disputes, emotionally sensitive messages, legal threats, or low-confidence answers to people. Longer prompts or a stronger model are not substitutes for escalation routes, review queues, operator interfaces, and records of approvals or changes. section 9.1.6
Long-running workflows can checkpoint state before a sensitive operation, notify a reviewer, and resume after approval without repeating earlier steps. LangGraph supports this style of stateful execution through checkpointers and graph review points. section 5.4.5 section 10.4.1
Collaborative story writing
A writing assistant generates a paragraph and pauses for user input. The user can approve it to continue, provide a correction to guide revision, or stop the loop. This illustrates feedback in the execution flow without implying that every use case needs a high-stakes approval gate. section 9.4.4
Human review adds latency and can overwhelm operators if escalation is unbounded. Set clear escalation policies, sampling rates, service-level agreements, and audit interfaces so review capacity is directed where it matters. section 9.2



























