Data quality is fitness for a specific use. In agentic systems, the test is whether information is fit to influence an agent for a particular purpose and level of autonomy. Weak data can become a control failure when an agent turns it into a claim, recommendation, disclosure, or action. section 5.6
Information can gain authority as it passes through a workflow. In Mata v. Avianca, generated legal authorities that did not exist were incorporated into a court filing. Plausibility survived selection and approval, but provenance, validity, applicability, and evidential support had not been established. section 5.0
Assess quality against the decision surface, not just the underlying records:
- Accuracy: Verify both source values and the agent’s interpretations. A valid rule can still be applied to the wrong situation, or an inference presented as fact. section 5.6
- Completeness: Check that required sources, evidence, exceptions, and policy inputs are present for the decision. section 5.6
- Timeliness and consistency: Check effective dates, freshness, supersession, and index lag; detect conflicting values, duplicates, and incompatible relationships. section 5.6
- Traceability: Connect sources and transformations to retrieval, model and tool activity, policy decisions, and the final decision episode. section 5.6
- Privacy and confidentiality: Discover and classify sensitive information, then mask, redact, or restrict it as appropriate. section 5.6
A legal decision surface
For a disputed period, verify that the agent uses the governing version of the relevant material, preserves delivery records, attributes the call transcript correctly, and links each legal proposition to an authority that supports it. These checks address both source fidelity and the conclusions drawn from sources. section 5.6
Table 5.6 Verifying and managing agentic data quality
|
Quality dimension
|
What you should verify and manage
|
|
Accuracy
|
Reconcile source data with authoritative records, test
transformation fidelity, and run versioned evaluation datasets against
retrieval, grounding, factual claims, tool use, and expected outcomes.
|
|
Completeness
|
Test whether required fields, sources, evidence,
exceptions, and policy inputs are present for the decision surface.
|
|
Timeliness
|
Monitor freshness, effective dates, supersession, pipeline
delay, retrieval-index lag, and context expiry.
|
|
Consistency
|
Detect conflicting values, duplicates, incompatible
formats, broken relationships, and disagreement across sources.
|
|
Traceability
|
Connect sources, transformations, retrieval, model and
tool activity, policy decisions, and the final decision episode.
|
|
Privacy and confidentiality
|
Discover, classify, mask, redact, and restrict personal,
privileged, confidential, or regulated information.
|
|
Compliance
|
Evaluate policy conditions, processing restrictions,
evidence requirements, approval routes, and control outcomes.
|
Use the verification table to translate quality dimensions into checks for a decision surface. section 5.6














