liveBook · Manning

Chapter 6: Ethical and Trustworthy GenAI

What you will learn in chapter 6 of AI Governance

The introduction explains why trustworthy GenAI requires ethical safeguards beyond security and privacy, with controls scaled to potential harm and maintained throughout the product’s life.

After this chapter you will be able to

What it builds on

Section by section

Introduction

The introduction explains why trustworthy GenAI requires ethical safeguards beyond security and privacy, with controls scaled to potential harm and maintained throughout the product’s life.

Key terms: ethics and trustworthiness, security and privacy, bias, ethical standards, design requirements, auditable controls, monitoring, technical debt, ethical risk, scrutiny

6.1 Accuracy & Reliability

The section explains why GenAI reliability is difficult to establish, distinguishes three ways it can fail, and connects those failures to real-world harm and governance.

Key terms: reliability failure, edge case, failure modes, hallucination, robustness, consistency of answers, ClaimAssist, unreliable decisions, governance controls, high-risk systems

6.1.1 Truthfulness & Hallucinations

This section explains why organizations remain accountable for hallucinations and how layered controls can reduce, detect, and contain them.

Key terms: AI hallucination, accountability, non-zero hallucination rate, reduction, detection, and containment, SaaS, retrieval stack, model selection, Retrieval-Augmented Generation (RAG), authoritative sources, RAG knowledge base

6.1.2 Robustness

This section explains how ordinary changes in wording, communication style, and input quality can make GenAI answers unreliable, and how teams can test and reduce that brittleness.

Key terms: robustness, hallucinations, ClaimAssist, sensitivity to phrasing and style, sensitivity to who is speaking, ambiguity and noise, OCR artifacts, equal treatment, explanation, contestability

6.1.3 Consistency of Answers

This section explains why LLM answers can vary across equivalent cases and how to make decisions more consistent, especially by routing rule-based questions to deterministic tools.

Key terms: token-prediction models, inconsistency, consistency, fair treatment, operational risk, temperature, random seed, model version, determinism, Chain-of-thought (CoT) prompting

6.2 Bias and Fairness

ClaimAssist shows how fixing hallucinations can still leave an AI system producing biased decisions.

Key terms: algorithmic bias, historical data, RAG

6.2.1 Real-world incidents

Real hiring and healthcare incidents show how AI can reproduce historical inequities, while training on AI-generated material can further erode coverage of underrepresented groups.

Key terms: bias, LLM-based hiring tools, health care costs, proxy, health needs, model collapse, underrepresented groups

6.2.2 What “fair” actually means

This section explains why fairness must be defined for a system’s particular harms, then tested and documented using relevant fairness lenses.

Key terms: fairness lenses, group fairness, individual fairness, equality, ClaimAssist, Responsible AI Charter, fairness goals, documentation and evidence

6.2.3 A practical bias-testing loop for LLM-based systems

This section explains how to test an LLM system for bias through repeatable black-box scenarios, group outcome comparisons, release checks, and contextual review.

Key terms: structured ML pipelines, retraining, black box, repeatable testing loop, critical journeys, paired scenarios, sensitive signal, outcome label, decision label, outcome rate per group

6.2.4 Bias controls

This section explains how to turn a use-case-specific definition of fairness into layered controls, measurement, human review, and appeals for GenAI systems.

Key terms: fair, model cards, public evaluation datasets, prompt design, training or reference data, protected attributes, proxy variables, output monitoring, keyword and pattern matching, human adjuster

6.3 Overreliance

This section explains how overreliance on AI turns nominal human review into rubber-stamping and how workflow design and monitoring can keep oversight meaningful.

Key terms: overreliance, automation bias, errors of commission, errors of omission, human oversight, human-in-the-loop, moral crumple zone, deliberate friction, supporting and conflicting evidence, original documents

6.3.1 Overreliance Controls

This section explains how to reduce overreliance on AI by making human review independent, practical, and measurable rather than a nominal approval step.

Key terms: accountability, AI assistance, human-only, workflow, training, KPIs, minimum review time, rubber-stamping, Assistant’s suggestion, Draft reasoning

6.4 Transparency and Explainability

The section distinguishes transparency from explainability and shows how a per-decision record supports investigations, explanations, and disclosure.

Key terms: transparency, explainability, AI-assisted decision, Decision Record, retrieval_sources, model, review_id, evidence, disclosure, AI bot

6.4.1 Transparency controls

This section shows how to assemble a Decision Record across provider, system, and reviewer layers, then use it for meaningful review and external transparency without mistaking logs for proof of a model’s reasoning.

Key terms: Decision Record, provider-layer logging, provenance, model card, versioned identifier, system-layer transparency, retrieval sources, source IDs, input transformations, retention schedules

6.5 Safety and Toxicity

This section explains how AI outputs can be unsafe despite sound decision logic, why such failures matter at scale, and how teams should identify the harms relevant to their deployment.

Key terms: unsafe or toxic outputs, safety failures, Decision Record, safety failure types, risk appetite, hard violations, soft violations, unsafe responses, guardrails, shared benchmark

6.5.1 Safety Controls

This section explains how to build safety controls across the input, output, response, and improvement stages of a GenAI system.

Key terms: system prompt, RAG, retrieval sources, training data, defense in depth, untrusted component, trusted pipeline, logging, audit trail, guardrails

6.5.2 Red Teaming in Practice

This section explains how diverse red teams find failures standard QA misses and turn those findings into ongoing tests and remediation decisions.

Key terms: red-teaming, adversarial inputs, standard QA, ClaimAssist, red team, domain experts, ethicists, affected communities, structured harm categories, regression suite

6.6 IP and Copyrighted Material

Introduces three points where GenAI IP risk arises and shows, through ClaimAssist, the cost of reproducing publicly available but unlicensed material.

Key terms: IP risks, training, users submit, model produces, ClaimAssist, licensed for commercial redistribution, infringing corpus, upfront licensing

6.6.1 Provenance & license Risk

This section explains how to establish rights to data before using it in GenAI systems and how to preserve the records and design choices needed to manage later IP claims or deletion requests.

Key terms: provenance, licensing, training corpus, fine-tuning, RAG, fair use, data register, AI inventory, licenses, vendor warranties

6.6.2 During Use: Sharing IP and Legal Protections

This section explains how sharing material with AI providers can put intellectual property and legal protections at risk, and how to control data flows and vendor use.

Key terms: fine-tuned model, resulting weights, proprietary know-how, attorney-client privilege, trade-secret status, patentability, confidentiality obligations, labeling scheme, intake process, sensitivity labels

6.6.3 After Release: Outputs, Leakage, and Distilled Models

This section explains how released GenAI outputs can create licensing exposure or enable model copying, and how organizations can reduce those IP risks.

Key terms: RAG, copyleft, open-source laundering, fine-tuned, API, distilled model, terms of service, IP indemnity, Legal and Procurement, open-weight models

6.7 Ethics in Agentic and Multi-Agent Deployments

This section explains how agentic actions and multi-agent handoffs create ethical harms that cannot be judged by inspecting individual answers alone.

Key terms: agentic AI, course of action, autonomy, misalignment, action irreversibility, loss of human agency, responsibility diffuses, miscoordination, conflict, collusion

6.7.1 Agentic AI controls

This section explains how to govern agents that can act by limiting autonomy, reviewing consequential actions before harm, recording and testing action trails, and expanding permissions only when evidence supports it.

Key terms: autonomy, point of harm, usable explanations, impact map, autonomy tiers, agentic identities, accountable owner, kill switch, plan preview, planning-mode profile

6.8 Summary

The chapter summary argues that trustworthy GenAI requires evidence for distinct, interconnected risks, effective human oversight, and sustained attention to harm beyond formal compliance.

Where this is used later

Concepts explained in this chapter

← Chapter 5: Privacy Chapter 7: Regulation and Compliance →

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.