liveBook · Manning

Human-in-the-loop

As explained in Understanding AI

Human oversight is the involvement of people in monitoring, evaluating, and controlling an AI system's outputs or behavior.

When an AI system produces an answer, should you act on it immediately? Human oversight means reviewing and judging AI outputs instead of treating them as automatically trustworthy. This matters because language models follow instructions, including harmful or misleading ones, and even strong prompts cannot replace human judgment. section 3.4.3 section 3.5

A practical oversight routine is straightforward:

  • Treat documents, websites, emails, and other external content as untrusted input. section 3.4.3
  • Review important outputs before acting on them, and maintain human oversight for high-stakes decisions. section 3.4.3
  • Use trusted tools and sources whenever possible. section 3.4.3
  • Do not assume that every instruction encountered by a model should be followed. section 3.4.3

An instruction hidden in an email

Imagine a model reading an email and producing a recommendation. The email may contain instructions designed to redirect the model, so the email is treated as untrusted content. A person checks the recommendation against trusted information before taking action. The model can accelerate the task, but the person remains responsible for deciding whether the result is safe and useful. section 3.4.3

How Understanding AI covers it

How deep it goes
Provides a practical checklist for reviewing outputs but does not specify a detailed evaluation framework or technical control mechanism.
Examples
Uses external documents, websites, emails, trusted sources, output review, and high-stakes decisions as oversight scenarios.
What it assumes
Establishes that language models follow instructions but cannot be trusted to judge whether every instruction is beneficial.
What you can do after
Evaluate AI-generated outputs critically before using them, particularly when decisions carry significant consequences.
How it connects
Connects human oversight with prompt injection, jailbreaking, hallucinations, overconfidence, bias, prompting, and language models.

Also explained in

Compare how 28 books explain human-in-the-loop →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Understanding AI. The section links point to the parts of the book it draws on.