An AI-assisted decision can leave a customer without a meaningful explanation if the system has no record of the model version, retrieved documents, or transformations applied to the output. Interpretability addresses the need for an understandable account of an outcome; in practice, that requires distinguishing an explanation from the evidence available to check it. section 6.4 section 6.4.1
Transparency reconstructs what happened: the inputs, configuration, information, tools, outputs, and people involved. Explainability asks why a particular outcome happened to a particular person. A complete trace may not explain the outcome, while a coherent rationale may lack records that allow it to be checked. section 6.4 section 6.4.1
Build an operational basis for reviewing decisions:
- Assemble a decision record from the execution pipeline, retrieval layer, and review interface. Include the model version, prompt configuration, retrieved sources, input, output, and human review as appropriate to the use case. section 6.4
- Expose supporting evidence, such as retrieved documents, policy clauses, and extracted facts, alongside known limitations. Reviewers can then assess the rationale rather than accept a black-box judgment. section 6.4.1
- Correlate prompts, retrieval, tool calls, approvals, and outputs with observability tooling. Instrumentation helps reconstruct events, but auditor-ready, tamper-proof evidence bundles typically require additional work. section 6.4 section 6.4.1
Table 6.3 The Decision Record (conceptual schema). Each row is one automated or AI-assisted decision. Fields are grouped by transparency layer. In practical scenarios, operational and regulatory information may live in separate tables or collections.
|
Layer
|
Field Name
|
Description
|
|
Provider
|
model
|
Identifier
or alias for the base model version. Contains {provider, model_name,
model_tag, api_release_date, model_card_url}.
|
|
System
|
system_prompt_version
|
Hash or ID
for the current prompt or system configuration
|
|
System
|
retrieval_sources
|
IDs or
hashes of documents shown to the model. Contains {doc_id, snippet_offset,
source_version}
|
|
System
|
input
|
The raw user
input that triggered the decision. Contains {user_prompt_redacted_id,
user_prompt_hash}
|
|
System
|
output
|
The raw
model output before post-processing. Contains {outcome (one of:
“approve”,“deny”,“request_info”,“escalate”), structured_key_facts:[],
policy_refs:[]}
|
|
System
|
postprocessing_id
|
List of
transformations applied after model response
|
|
Explainability
|
rationale_explanation
|
Human-readable
justification, constructed via template or retrieval
|
|
Explainability
|
evidence
|
List of
factors that influenced the decision, derived from business rules, retrieval
sources, or structured extraction. Contains {factor_id, factor_value, source,
actionable_guidance}
|
|
Explainability
|
confidence_flag
|
Optional
confidence or evidence score
|
|
Reviewer
|
review_id
|
Identifier
for the human review. Contains {reviewer_id, outcome (one of:
“approve”,“deny”,“request_info”,“override”), review_timestamp,
override_explanation}
|
|
Customer
|
access_log:[]
|
Logs access
requests. Contains {user_id (pseudonymized), timestamp, request_id}
|
|
Regulator
|
bundle_id
|
ID tying
this record to quarterly export bundles
|
|
Regulator
|
lawful_basis
|
GDPR basis
for retention and processing
|
|
Regulator
|
record_hash
|
Cryptographic
hash of the full record, logged for audit integrity
|
The decision record schema groups fields by transparency layer, providing a starting point for a correlated case file. section 6.4
A claim rationale is not proof of reasoning
A simulated trace recommends a partial denial because repair costs are below the deductible and damage falls under a policy exclusion. The rationale gives a reviewer concrete claims to check against the policy and evidence, but it does not establish that those factors actually caused the model’s output. Model-generated traces can be incomplete, strategic, or reconstructed after the fact. section 6.4.1
This distinction matters especially for autonomous agents: visible “thinking” does not guarantee a genuine explanation, even when the agent’s actions affect individuals. Keep human review connected to evidence and use override patterns to prioritize prompt, retrieval, and documentation fixes. section 2.6.4 section 6.4.1























