liveBook · Manning

Trust

As explained in Designing AI Agents

The confidence an organization grants an agent to act within defined limits, informed by its observed outcomes.

Trust determines how much room an agent receives to act inside a hard governance ceiling. That room can expand or contract with observed outcomes and the risk of the current task. Reliability on familiar, reversible work does not necessarily justify the same freedom on a new, irreversible task. section 9.1.1

A hard ceiling remains in force regardless of trust: it can prohibit access to resources, cap amounts, or require another authority for certain actions. Progressive trust changes working room within those limits; it does not waive an explicit denial. section 9.1.1 section 9.8

In Argus, a governance facade orders the tool-call path: approval gate, trust decision, policy envelope, and observation. An ASK decision is provisional: the trust check may waive human review at the current scoped level. At SUPERVISED, a medium-risk edit can proceed this way, while a high-risk command still asks and a critical external effect always asks. The outcome is recorded for the next trust decision. section 9.8

Human approval needs evidence, not just a request. The host retains a versioned review packet containing the objective, target, proposed change, evidence reference, and policy version. The callback displays the evidence and returns the approving reviewer’s identifier, or declines. section 9.8

Trust also shapes control placement. A guarded state-changing action sits below higher-trust controls: a pre-check decides whether it may start, a controlled executor limits it, and a post-check queries protected evidence before accepting the result. Keep policies and verifiers outside the agent’s writable state. section 6.5

The governance boundary protects only calls routed through it. The Argus checkpoint governs repository reads and lint actions, but other module calls still bypass the facade; a production build must route every operation with an external effect through the same boundary. section 9.8

Table 9.1 The autonomy spectrum. Five operational levels and a research horizon.
Level
Name
Agent role
Human role
Example
0
Observe
Suggests only
Decides and executes
Shadow-mode review
1
Assist
Reads, analyzes, and proposes
Reviews and acts
Read-only review assistant
2
Supervised
Executes routine actions
Approves higher-risk actions
Coding agent with approval
3
Autonomous
Handles bounded workflows
Reviews outcomes
Bounded background agent
4
Delegated
Operates within an explicit charter
Monitors and audits
Production agent within charter
5
Self-governing
Sets its own objectives
Audits periodically
Research horizon; not deployed

Compare the agent and human roles at each autonomy level; trust changes the room granted within the governing limits. section 9.1.1

How Designing AI Agents covers it

How deep it goes
Explains scoped approval routing, evidence retention, outcome recording, and the limits of a partially applied governance boundary.
Examples
Uses a code-review agent’s supervised tool calls and a compensation approval packet with current and proposed amounts.
What you can do after
Evaluate how task risk, hard ceilings, and governance coverage affect an agent’s permitted autonomy.
How it connects
Connects trust with autonomy, approval gates, policy envelopes, and guardrails.

Also explained in

Compare how 18 books explain trust →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Designing AI Agents. The section links point to the parts of the book it draws on.