liveBook · Manning

Policy check

As explained in Designing AI Systems

The checking of action parameters against business rules and domain constraints rather than only structural or type requirements.

A policy check evaluates whether a proposed action is permitted by business rules and domain constraints, not merely whether its structure or types are valid. It sits between model output and tool execution, using the requester, session state, prior actions, and tool arguments as context. section 6.6

Policy checks can enforce rules such as:

  • Requiring completed intake forms before appointment booking. section 6.6
  • Rejecting dates outside an allowed booking window or procedures not covered by a plan. section 6.6
  • Limiting bookings per session, restricting cancellations to 24 hours, or requiring approval above a refund threshold. section 6.6

Booking-flow check

After the model requests a booking tool, call a guardrails service with the booking policy, action name, session context, and decoded arguments. Block the call and report a violation when the policy denies it. section 6.6

Figure 6.10 The five-point inspection model establishes a defense-in-depth architecture for AI interactions. This multi-layered approach ensures comprehensive policy enforcement at every stage of the request lifecycle:(1) Input Validation intercepts malicious prompt injections or off-topic user messages before they reach the model. (2) Pre-Execution Policy evaluates the model's intended tool calls against session state and business rules (e.g., verifying prerequisites are met before booking). (3) Argument Validation ensures the specific parameters sent to external APIs strictly adhere to domain constraints and schemas.(4) External Response Filtering sanitizes sensitive raw data or handles errors returning from external systems before they are incorporated into the AI's context. (5) Output Filtering serves as the final, holistic gate to catch hallucinations, redact PII, and enforce brand safety before the assembled response is delivered to the user.

The five-point inspection model places policy checks before execution and argument validation, alongside input, response, and output safeguards. section 6.6

How Designing AI Systems covers it

How deep it goes
Explains contextual evaluation and contrasts policy errors with schema errors, without detailing policy implementation.
Examples
Uses appointment booking, referral matching, booking windows, coverage checks, session limits, cancellations, and refund approvals.
What it assumes
Assumes tool calls, arguments, external systems, and session context; distinguishes business validation from structural validation.
What you can do after
Operate guarded tool execution by evaluating actions and arguments against contextual business policies.
How it connects
Connects tool execution, schema validation, session state, guardrails, and defense in depth.

Also explained in

Compare how 2 books explain policy check →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Designing AI Systems. The section links point to the parts of the book it draws on.