liveBook · Manning

Accuracy

As explained in Architecting for Autonomy

Accuracy is the reliability of the premises, interpretations, propositions, and conclusions from which an agent reasons and acts.

Accuracy is the reliability of the premises from which an agent reasons and acts. It applies not only to source values but also to the interpretations, propositions, and conclusions derived from them. When an agent uses information to make or trigger a decision, inaccurate information can become a control failure. section 5.6

A classification accuracy score measures one part of this problem; it does not describe the consequences of errors. A production decision also needs an account of which cases fail, where they go, and who or what is affected. section 3.1 section 5.6

Claims triage: the unresolved 6%

An illustrative health-insurance pilot classified claims as routine, complex, or high-acuity with 94% accuracy on a held-out benchmark. It reduced average triage time from 40 minutes to 4 minutes, but the team had not estimated the cost of errors. A high-acuity claim misrouted as routine could wait 48–72 hours, potentially missing an authorization window and triggering complaints. The estimated cost of one such misrouting was $8,000–$50,000. The aggregate score alone could not establish whether production use was acceptable. section 3.1

Accuracy failures can occur even when a source is real or a rule is valid:

  • A summary can exaggerate what its source supports, or an agent can turn an inference into a fact. section 5.6
  • An agent can apply a valid rule to the wrong situation, attribute a statement to the wrong person, or cite an authority that does not support a proposition. section 5.6

Protect accuracy by using authoritative sources, evaluating retrieval and grounding, preserving transformation lineage and summarization fidelity, and linking claims explicitly to evidence. For a disputed period, verify that the governing version is used and that records and statements are faithfully attributed. section 5.6

Table 5.6 Verifying and managing agentic data quality
Quality dimension
What you should verify and manage
Accuracy
Reconcile source data with authoritative records, test transformation fidelity, and run versioned evaluation datasets against retrieval, grounding, factual claims, tool use, and expected outcomes.
Completeness
Test whether required fields, sources, evidence, exceptions, and policy inputs are present for the decision surface.
Timeliness
Monitor freshness, effective dates, supersession, pipeline delay, retrieval-index lag, and context expiry.
Consistency
Detect conflicting values, duplicates, incompatible formats, broken relationships, and disagreement across sources.
Traceability
Connect sources, transformations, retrieval, model and tool activity, policy decisions, and the final decision episode.
Privacy and confidentiality
Discover, classify, mask, redact, and restrict personal, privileged, confidential, or regulated information.
Compliance
Evaluate policy conditions, processing restrictions, evidence requirements, approval routes, and control outcomes.

Use the accuracy row as a verification checklist for source reconciliation, transformation fidelity, and evaluations of retrieval, grounding, claims, tool use, and outcomes. section 5.6

How Architecting for Autonomy covers it

How deep it goes
Defines accuracy beyond source values, identifies interpretation failures, and gives verification controls; the claims pilot examines consequences beyond an aggregate score.
Examples
Uses an illustrative claims-triage pilot and legal propositions tied to governing versions, delivery records, transcripts, and supporting authorities.
What you can do after
Evaluate accuracy risks by checking source and claim fidelity and assessing the downstream consequences of classification errors.
How it connects
Connects accuracy to hallucination, grounding, retrieval evaluation, transformation lineage, and evidence attribution.

Also explained in

Compare how 49 books explain accuracy →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Architecting for Autonomy. The section links point to the parts of the book it draws on.