liveBook · Manning

Validation layer

As explained in AI Governance

Input/output validation is the inspection of AI requests and responses to sanitize them or detect content that violates defined requirements.

An input/output validation layer sanitizes requests and responses before they reach an external LLM API. When the vendor operates it, the layer forms a trust boundary: organizations typically cannot directly enforce or change its rules, so they must assess how well it detects prompt injections, harmful content, accidental data leakage, and hallucinations. section 2.2

Assess vendor controls and compensate for gaps:

  • Request documentation of validation processes; ask how prompt injection and harmful content are detected, and how related logs are used and retained. section 2.2
  • Where vendor controls fall short, add safeguards such as internal review, clear user warnings, or customized guardrails. section 2.2

How AI Governance covers it

How deep it goes
Explains placement, trust implications, due diligence, and added safeguards without specifying validation implementation.
Examples
Cites false alarms in engineered systems, incorrect creditworthiness classifications, and misleading disease-diagnosis guidance.
What you can do after
Evaluate vendor controls by requesting process documentation and clarifying detection practices and related log handling.
How it connects
Connects validation to prompt injection, harmful content, data leakage, hallucinations, vendor due diligence, and guardrails.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of AI Governance. The section links point to the parts of the book it draws on.