How deeply does Building Reliable AI Systems cover human-in-the-loop?

Specifies legal review safeguards and agent approval steps, but gives no detailed human-review workflow implementation.

Human-in-the-loop

As explained in Building Reliable AI Systems

Human supervision of an AI system's outputs, decisions, and operation.

Human-in-the-loop supervision places people in the path of AI decisions or outputs. In high-stakes workflows, require human validation before outputs are acted on; production agents can pause for approval and resume execution. section 1.2.1 section 6.2.2

Use review where plausible output errors carry consequential risks:

Legal work: verify AI-generated content and citations against authoritative sources, and require human review before output reaches a court. section 1.2.1
Code generation: review complex logic carefully; pattern matching and boilerplate generation do not guarantee correctness. section 5.2.3
Production agents: use approval steps that can pause and resume agent execution. section 6.2.2

How Building Reliable AI Systems covers it

liveBook manning.com

Read Building Reliable AI Systems on liveBook Get the book at manning.com

Also explained in

liveBook manning.com

Introduction to Generative AI, Second EditionHuman-in-the-loop is a system-design pattern in which people participate in monitoring, evaluating, correcting, or governing an AI system.
liveBook manning.com

The Art of AI Product DevelopmentHuman evaluation is the assessment of model outputs by people using qualitative or task-specific judgments.
liveBook manning.com

AI Agents in Action, Second EditionHuman evaluation is the assessment of system outputs or behavior by people applying judgment that may depend on context, policy, tone, or edge cases.
liveBook manning.com

Generative AI in ActionHuman evaluation is the assessment of an artifact or system by people using judgments, observations, surveys, or interviews.
liveBook manning.com

Starting Data Analytics with Generative AI and PythonOversight is the monitoring and supervision of an AI system's performance, impacts, and compliance by responsible people or groups.
liveBook manning.com

Machine Learning System DesignJudgments or preference signals supplied by people about the quality or desirability of system outputs.
liveBook manning.com

Build an AI Agent (From Scratch)Human evaluation is the direct assessment of system outputs by professional graders or domain experts.
liveBook manning.com

AI-Powered SearchA judgment is a human assessment of the relevance of a search result to a query or information need.
liveBook manning.com

AI Agents and Applications
liveBook manning.com

"Looks Good to Me"The human capacity to evaluate software decisions using nuance, context, history, and broader situational understanding.
liveBook manning.com

Digital Twins in ActionHuman oversight is the supervision and authorization of an autonomous system's decisions and actions by people.
liveBook manning.com

Data-Driven CybersecurityHuman oversight is the review, supervision, or intervention by people in an automated system's operation and decisions.
liveBook manning.com

Financial AI in PracticeHuman oversight is the involvement of people in reviewing, approving, or supervising system decisions and operation.
liveBook manning.com

Effective Conversational AIHuman review is the examination of system outputs by people who assess their accuracy and contextual appropriateness.
liveBook manning.com

First-Party Data ActivationA human-in-the-loop model is an AI system or workflow in which people participate in reviewing, approving, or directing model outputs.
liveBook manning.com

Grokking StatisticsHuman in the loop is a system pattern in which people participate in reviewing, labeling, guiding, or validating automated processing.

Compare how 17 books explain human-in-the-loop →

Related concepts

evaluatoralternative to · An evaluator is a model or component that assesses outputs against specified criteria and produces judgments, scores, comparisons, or feedback.
AI agentused by · An AI agent is a computational system that uses artificial intelligence to interpret information, select actions, and pursue goals through interaction with an environment.
language modeluses · A language model is a computational model that represents patterns in linguistic tokens to estimate sequence probabilities and predict or generate language from context.
automationcontrasts with · Automation is the execution of tasks or processes by machines or software with reduced direct human intervention.

This explainer was generated by Manning's knowledge graph from the text of Building Reliable AI Systems. The section links point to the parts of the book it draws on.