Unsafe content is material that can cause or materially contribute to serious physical, psychological, social, or other harm. On a platform, this can include abuse, spam, scams, illegal material, hate messages, misinformation, and content promoting eating disorders. A recommender system may amplify such material when engagement signals reward emotionally charged or divisive posts. section 3.1 section 1.3
Unsafe content becomes especially consequential when platform systems and social conditions increase its reach:
- Engagement-optimized feeds can make hateful, divisive, or emotionally charged posts more likely to spread widely. section 7.3
- Personalized delivery can repeatedly expose vulnerable people to material that reinforces an existing struggle, such as eating-disorder content. section 1.3
- Weak moderation can make a platform less welcoming, reduce user trust, and allow harmful material to remain visible. section 3.1
- The same distribution technology can expand positive movements as well as harassment campaigns, conspiracy theories, and other harmful activity. section 7.6
Myanmar and anti-Rohingya content
Facebook became a major way to access the internet, consume news, and form a worldview in Myanmar. Anti-Rohingya posts spread through engagement-focused recommendations, while activists’ requests for stronger moderation received little response. The resulting information pollution contributed to real-world ethnic cleansing and genocide risks, showing how unsafe content can move from online circulation to physical harm. section 7.3 section 7.6
