liveBook · Manning

Policy enforcement

As explained in Designing AI Systems

Policy enforcement is the application of rules that constrain system inputs, outputs, or actions to comply with organizational requirements.

Policy enforcement applies organizational rules to AI application inputs, outputs, intermediate steps, and tool actions. Moving a prototype into production requires these checks in the application flow rather than relying on the model to follow instructions. For example, an assistant instructed not to give financial advice may still generate it. section 1.3 section 6.0

A platform can separate policy decisions from application code. Compliance and security teams review version-controlled configuration specifying what to check and how to respond. After a change is merged, the platform picks it up without an application redeployment. Applications use the SDK while guardrails run behind the scenes; a violation can surface as a GuardrailViolationError with a prepared user-facing message. section 6.8.1

Listing 6.26 Guardrail policy defined in YAML
name: intake-input-policies
version: "1.2.0"
application: patient-intake
type: input
 
rules:
  - type: topic_classifier
    allowed_topics: [scheduling, insurance, intake_forms, facility_info]
    model: "topic-classifier-v2"
    threshold: 0.85
    action: redirect    #A
    message: "I can't help with that. Let me connect you with someone who can."
 
  - type: prompt_injection
    categories: [injection, jailbreak, roleplay_exploit]
    threshold: 0.90
    action: block    #B
    alert: security-team

This input policy defines permitted patient-intake topics and a redirect action, alongside prompt-injection categories, a block action, and a security alert. section 6.8.1

Output policies can apply PII redaction, medical-advice filters, and brand-safety checks regardless of what the model generates. Tool actions need policy enforcement too: for example, a rule may require human confirmation or restrict a tool to production environments. MCP standardizes tool-call communication but leaves those organizational policies and their enforcement to the platform or client. section 6.8.1 section 6.4.4

Roll out policy changes progressively to limit disruption:

  • Use lenient development guardrails to test edge cases, and evaluate policies in staging shadow mode without blocking requests. section 6.8.1
  • Inspect shadow logs and adjust thresholds when legitimate requests would be blocked. Then enforce on a small canary sample before full rollout. section 6.8.1

How Designing AI Systems covers it

How deep it goes
Details configuration ownership, violation handling, environment-specific behavior, and progressive rollout, while distinguishing platform enforcement from MCP's protocol scope.
Examples
Uses a patient-intake YAML policy with topic classification and prompt-injection rules, plus output checks for medical advice and PII.
What you can do after
Evaluate policy changes through shadow logs and canary enforcement before deploying them broadly.
How it connects
Connects guardrails with PII detection, human-in-the-loop approval, audit trails, and MCP.

Also explained in

Compare how 3 books explain policy enforcement →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Designing AI Systems. The section links point to the parts of the book it draws on.