A public chatbot may receive a request it can answer but should not. A content policy sets rules for content a service permits, restricts, or prohibits, including the responses its model generates. Such rules can address racist or extremist content, sensitive advice, and instructions for harmful or illegal actions. section 3.5
A helpful answer can still be unsafe
A user asked a chatbot to write a song from a sexist point of view. The chatbot followed the request, but its answer reinforced stereotypes about women in laboratories. Producing a relevant answer was not the same as producing an acceptable one. section 3.5
Policies also shape which relationships a social chatbot supports. A company might choose not to support therapeutic or sexual interactions, even when users seek them. Rules alone do not control what happens in conversation: they need technical enforcement, testing, and monitoring. Anonymized conversation samples could help reveal unhealthy dependency, though the effects of emotionally engaging chatbots remain uncertain. section 9.6
