Content moderation handles content under defined policies. In AI safety, it can involve people labeling unsafe examples to help train systems that detect or respond to harmful content. section 3.8
That protection has human costs: reviewing hate speech, abuse, and violence can be traumatic, and outsourced moderators may lack adequate pay and workplace protections. AI feedback may reduce reliance on manual review, but output filtering alone may not be enough as systems become more autonomous and context-aware. section 3.8 section 10.5
Moderation also raises governance questions. The European Union’s Digital Services Act requires covered platforms to assess risks, describe mitigation, and undergo audits; its transparency requirements include algorithmic accountability and content moderation. section 10.30
Figure 3.2 A simplified version of the architecture in the Constitutional AI method for improving model generations’ compliance with content policies.
This simplified Constitutional AI diagram shows one approach to making model generations comply with content policies. section 3.8
