liveBook · Manning

Progressive trust level

As explained in Designing AI Agents

A level of confidence in an agent that determines how much routine approval work may be removed.

A trust level represents confidence in an agent and changes how an Approval Gate routes routine review; it does not grant tool permissions. Levels range from Observe and Assist through Supervised to Autonomous and Delegated. section 9.4.5

Adjust trust from recorded outcomes, not a single favorable result:

  • Track action totals, successes, overrides, errors, and consecutive errors. section 9.4.5
  • Promote only when configured action-count and performance thresholds are met. Consecutive errors trigger a demotion attempt and consume current evidence before promotion is checked. section 9.4.5
  • Treat sample thresholds as teaching examples; production promotion also needs task-coverage rules, a risk-appropriate observation period, owner approval, and transactional state stored outside the agent’s write scope. section 9.4.5
Listing 9.8 Trust levels and metrics (patterns/progressive_commitment.py)
from dataclasses import dataclass
from enum import IntEnum
 
class TrustLevel(IntEnum):
    OBSERVE = 0
    ASSIST = 1
    SUPERVISED = 2
    AUTONOMOUS = 3
    DELEGATED = 4
 
@dataclass
class TrustMetrics:
    total_actions: int = 0
    successful_actions: int = 0
    user_overrides: int = 0
    errors: int = 0
    consecutive_errors: int = 0             #A
 
    @property
    def success_rate(self) -> float:
        return (
            self.successful_actions / self.total_actions
            if self.total_actions else 0.0
        )
 
    @property
    def override_rate(self) -> float:
        return (
            self.user_overrides / self.total_actions
            if self.total_actions else 0.0
        )
 
    @property
    def error_rate(self) -> float:           #B
        return (
            self.errors / self.total_actions
            if self.total_actions
            else 0.0
        )
 
@dataclass
class EscalationThresholds:
    min_actions: int = 50
    min_success_rate: float = 0.95
    max_override_rate: float = 0.05
    max_error_rate: float = 0.02
    consecutive_errors_for_demotion: int = 3

The listing defines the trust levels and outcome metrics used by the progression logic. section 9.4.5

Keep fixed human boundaries

Critical operations, insufficient evidence, and ethically sensitive actions still require human review, even at the highest trust level. Trust can reduce routine approvals, but cannot override these triggers or the blast-radius ceiling. section 9.4.2

How Designing AI Agents covers it

How deep it goes
Defines five levels and implements metrics, asymmetric promotion and demotion, and approval routing; leaves production promotion safeguards to deployment.
Examples
Uses critical operations, missing evidence, and ethical sensitivity as cases that keep human review mandatory.
What it assumes
Assumes an approval gate, risk classification, and explicit human-sovereignty boundaries.
What you can do after
Operate approval routing by recording outcomes, applying performance thresholds, and preserving mandatory human-review triggers.
How it connects
Connects approval gates, human sovereignty, blast-radius control, observability, and regression testing.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Designing AI Agents. The section links point to the parts of the book it draws on.