liveBook · Manning

Uncertainty

As explained in Architecting Autonomous Agentic Systems

A condition in which the outcomes, costs, benefits, or behavior of a system cannot be known with complete confidence in advance.

Uncertainty is the condition in which a system’s outcomes, costs, benefits, or behavior cannot be known with complete confidence in advance. In autonomous systems, it matters at the point where an enterprise delegates decisions: an agent may act on unreliable predictions, while the costs and timing of its benefits remain uncertain. section 2.2.2 section 3.5.1

Uncertainty does not require choosing between unrestricted autonomy and shutting a system down. Explicit thresholds, decision traces, and human oversight can trigger a narrower boundary while a problem is investigated. The oversight role must also be clear: validating an agent’s protocol is different from owning the underlying clinical decision. section 2.2.2

Narrowing autonomy during model drift

A hospital agent’s uncertainty score rises from 0.12 to 0.34, crossing a 0.30 threshold. An operations reviewer checks the decision trace and temporarily reduces the agent’s authority from deferring up to four elective procedures to two for twelve hours. The change is logged for governance review while the model is investigated; clinical operations continue with a reduced automation boundary. section 2.2.2

Risk also reshapes the business case. Validation and review affect failure probability, while tighter authority boundaries limit potential blast radius but require more escalations and manual reviews. Benefits may arrive more slowly than projected as trust develops and operating practices stabilize. These effects must be considered together rather than estimated in isolation. section 3.5.1

Three mechanisms make economic uncertainty explicit:

  • Expected failure cost combines failure probability with cost per event, exposing potential decision cascades and regulatory penalties during scoping. section 3.5.2
  • Timing sensitivity tests whether a business case remains viable if trust and governance take longer to stabilize. A twelve-month path to breakeven is more fragile than a six-month path, even with identical steady-state returns. section 3.5.2
  • Benefit realization curves represent a nonlinear learning period rather than assuming that full value appears immediately. Trust, governance maturity, and operating practices develop alongside one another. section 3.5.2

Use these mechanisms before committing to construction, and pair them with operational thresholds and named oversight responsibilities. A viable design must tolerate both uncertain financial outcomes and uncertain behavior within its delegated authority. section 3.5.2 section 2.2.2

How Architecting Autonomous Agentic Systems covers it

How deep it goes
Applies thresholds and three risk mechanisms, including expected failure cost and timing sensitivity, to oversight and business-case decisions.
Examples
Uses a hospital agent’s rising uncertainty score and a financial-services data-handling error to illustrate operational and autonomy-scope risks.
What you can do after
Evaluate uncertain autonomous initiatives using failure cost, benefit timing, realization curves, and explicit oversight boundaries.
How it connects
Connects uncertainty to human oversight, model drift, governance, delegated authority, failure probability, and benefit realization.

Also explained in

Compare how 17 books explain uncertainty →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Architecting Autonomous Agentic Systems. The section links point to the parts of the book it draws on.