When an AI-assisted decision is challenged, scattered logs may not show which model, documents, and review produced the outcome. A decision record combines those events into one structured case file for each decision, so the decision can be reconstructed and investigated. section 6.4 section 6.8
The record supports transparency, but it is not itself an explanation. A record can establish what information and configuration were used; explaining why a particular outcome occurred may require further investigation. Conversely, a fluent rationale is not proof without records that can be checked. section 6.4
Build the case file from correlated pipeline, retrieval, and review events:
- Identify the model provider and version, prompt or configuration version, and any intermediary settings. Version identifiers help investigators distinguish decision behavior from later vendor updates. section 6.4.1
- Record the input, model output before post-processing, and identifiers or hashes for retrieved documents and their versions. Link events with a shared request identifier. section 6.4 section 4.3.1
- Capture the evidence relied on, guardrail outcomes, and any human review, including overrides and reviewer reasoning. These give an affected person or investigator something concrete to examine. section 6.4 section 4.3.1
- For agents, capture planning steps, tool selections, and intermediate conclusions as well as outputs. These traces help investigate unexpected actions, instruction failures, or prompt injection. section 4.8.8
Table 6.3 The Decision Record (conceptual schema). Each row is one automated or AI-assisted decision. Fields are grouped by transparency layer. In practical scenarios, operational and regulatory information may live in separate tables or collections.
|
Layer
|
Field Name
|
Description
|
|
Provider
|
model
|
Identifier
or alias for the base model version. Contains {provider, model_name,
model_tag, api_release_date, model_card_url}.
|
|
System
|
system_prompt_version
|
Hash or ID
for the current prompt or system configuration
|
|
System
|
retrieval_sources
|
IDs or
hashes of documents shown to the model. Contains {doc_id, snippet_offset,
source_version}
|
|
System
|
input
|
The raw user
input that triggered the decision. Contains {user_prompt_redacted_id,
user_prompt_hash}
|
|
System
|
output
|
The raw
model output before post-processing. Contains {outcome (one of:
“approve”,“deny”,“request_info”,“escalate”), structured_key_facts:[],
policy_refs:[]}
|
|
System
|
postprocessing_id
|
List of
transformations applied after model response
|
|
Explainability
|
rationale_explanation
|
Human-readable
justification, constructed via template or retrieval
|
|
Explainability
|
evidence
|
List of
factors that influenced the decision, derived from business rules, retrieval
sources, or structured extraction. Contains {factor_id, factor_value, source,
actionable_guidance}
|
|
Explainability
|
confidence_flag
|
Optional
confidence or evidence score
|
|
Reviewer
|
review_id
|
Identifier
for the human review. Contains {reviewer_id, outcome (one of:
“approve”,“deny”,“request_info”,“override”), review_timestamp,
override_explanation}
|
|
Customer
|
access_log:[]
|
Logs access
requests. Contains {user_id (pseudonymized), timestamp, request_id}
|
|
Regulator
|
bundle_id
|
ID tying
this record to quarterly export bundles
|
|
Regulator
|
lawful_basis
|
GDPR basis
for retention and processing
|
|
Regulator
|
record_hash
|
Cryptographic
hash of the full record, logged for audit integrity
|
Use the conceptual schema to see how fields are grouped across transparency layers; operational and regulatory information may be stored separately. section 6.4
Examining a claims decision
A simulated claims rationale recommends a partial denial because a $500 repair is below a $1,000 deductible and the damage is described as cosmetic wear and tear. A decision record can connect that output to the model version, retrieved policy sources, and any review, so those factors can be checked rather than accepted solely because the rationale sounds plausible. section 6.4.1 section 6.4
Instrument collection from the start with an observability platform such as Langfuse, Arize Phoenix, or LangSmith. These tools provide a foundation, not automatically tamper-proof, auditor-ready evidence bundles. Because inputs and outputs may contain sensitive data, establish a lawful basis, retention limits, and access controls; sanitize access logs where appropriate. section 6.4 section 6.4.1 section 4.3.1

