A cascade of trust occurs when an agent passes on a tentative finding and the next agent treats it as fact. Uncertainty disappears at each handoff: a tentative finding becomes “the policy says,” or a guess about intent becomes the basis for action. section 6.7
Errors compound across rewrites
In one study, GPT-3.5 repeatedly rewrote news articles under different stylistic prompts. After eight rewrites, correct answers about the articles fell from about 96% to under 60%, illustrating how successive processing can compound errors. section 6.7
Table 6.6 How agentic AI amplifies existing ethical and trustworthiness risks
|
Chapter 6 risk
|
Non-agentic failure
|
What changes in agentic systems
|
|
Accuracy and hallucinations
|
The model states a made-up fact, clause, case, or source.
A person reads it and decides whether to trust or use it.
|
The model treats its own invented fact as true and builds
on it, passing it forward or storing it as memory that resurfaces in official
documents.
|
|
Robustness
|
A typo, an odd phrasing, or a dialect changes the wording
of the answer, and the change is visible in the text a person reviews.
|
The same small change makes the model pick a different
action: different documents, a different tool, or skipping the escalation
step entirely.
|
|
Consistency
|
Two similar customers get differently worded answers to
the same question.
|
Two similar customers are routed differently: one gets
extra checks, the other a fast decision. The gap is in treatment, not
wording.
|
|
Bias and fairness
|
The model gives weaker answers for some groups or repeats
bias from its training data. Even when it pulls documents first, the bias
lands in the one answer a reviewer can see.
|
Bias compounds across steps: which claims the agent
retrieves, which it treats as the priority case, and which action it takes.
No single step looks wrong, so the pattern is hard to trace.
|
|
Overreliance
|
A human reviewer may negligently accept a faulty AI output
which causes harm.
|
The reviewer sees only the finished result, not the steps
behind it. By the time they look, the action is already taken.[129]
|
|
Transparency and explainability
|
The organization cannot trace which documents shaped an
answer, what is the confidence of that answer, and whether the answer was
reviewed by a human.
|
The organization cannot trace how the agent planned the
work, which tools it used, or which resources it accessed. The model chose
most of it itself.
|
|
Safety and toxicity
|
When prompted, the model produces harmful, manipulative,
or inappropriate answers, and a person sees it before it goes anywhere.
|
The model bypasses HITL, for example having a subagent approve its own action
and then use that approval as if it were from a human[130].
|
|
IP and provenance
|
Carelessly providing licensed data as input to a model or
using the model’s outputs can damage an organization’s intellectual property.
|
The agent may edit resources, combine multiple resources,
or share documents autonomously, threatening intellectual property and
muddying ownership.
|
Compare how an invented fact can be treated as true and passed forward or stored as memory in an agentic system. section 6.7
