liveBook · Manning

Content filtering

As explained in Designing AI Systems

The detection or suppression of inappropriate, harmful, or otherwise disallowed content in model inputs or outputs.

Content filtering checks model inputs or outputs for disallowed material. It is one layer of defense in depth; it does not replace policies that control which actions an assistant may take. section 6.6 section 10.7

In an assistant workflow, filtering can happen at multiple points:

  • Check incoming messages before they reach the model; a topic classifier can block requests such as medical advice. section 10.7
  • Filter the model’s response before returning it; output checks can catch hallucinations or redact personally identifiable information. section 6.6
Figure 6.9 Execution policy guardrails control what the system does, and content guardrails filter what the model says. Both are necessary, but only execution policies can enforce business-specific rules.

Contrasts filtering what a model says with execution policies that control what the system does. section 6.6

How Designing AI Systems covers it

How deep it goes
Shows input blocking and output filtering in an agent workflow, but does not specify filter internals.
Examples
Uses a topic classifier to block medical-advice requests and identifies PII redaction as an output check.
What you can do after
Apply it via the Guardrails Service to validate inputs and filter model responses in an agent workflow.
How it connects
Links content filtering to input validation, output filtering, and execution policies.

Also explained in

Compare how 5 books explain content filtering →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Designing AI Systems. The section links point to the parts of the book it draws on.