A governance, risk management, and compliance (GRC) program coordinates how an organization identifies, controls, and responds to risks. For generative AI, it connects governance decisions to engineering safeguards and operational monitoring. It cannot eliminate hallucinations or false positives, but can help detect and correct them before they escalate. section 1.1
Document vetting under attack
A vendor uses an open-source LLM to check passport photos and government IDs for an airline. Hidden instructions in an uploaded image trick the model into bypassing checks, approving invalid documents and creating security, regulatory, and reputational consequences. section 1.1
Controls should span deployment and operation:
- Before deployment, test safety and bias, run adversarial prompt suites, and red-team the system. section 1.1
- After launch, use output filters, rate limits, live toxicity and drift metrics, and anomaly detection. section 1.1
Table 1.1 Why Generative AI Breaks Traditional Control Models (Continued)
|
Pre-deployment
assurance
|
Tests, code
review, static analysis, red-teaming (ethical hackers)
|
Role-specific
training, competency certification, scenario-based drills
|
Safety and
bias testing, adversarial prompt suites, red-teaming, interpretability tools
(e.g. probing)
|
|
Post-launch
controls
|
Logging,
health checks, intrusion detection, WAF, auto-rollback on error rate spike
|
Supervisors,
peer review, escalation protocols
|
Output
filters, rate limiting, live metrics on toxicity and drift, anomaly detection
on embeddings
|
|
Failure
detection & attribution
|
Clear (stack
traces, logs, input replay)
|
Mixed
(relies on human reporting or
observation)
|
Opaque
(failures can appear valid; root cause buried in model weights)
|
|
Memory
/ learning capability
|
None (unless
explicitly coded)
|
Present
(experience, judgment, learning over time)
|
Present
(training data, fine-tuning, context memory)
|
Compare pre-deployment assurance, post-launch controls, and failure attribution across software, human operators, and generative AI. section 1.1
