A social media feed can rank content according to user engagement. The system’s recommendations shape what people see; their interactions then provide validation and training data for later recommendations. section 8.0
This is a socio-technical loop, not just an algorithmic one: human responses, system behavior, and the society and historical period in which the technology emerges all help explain its effects. section 8.0
Engagement-based recommendations are considered a major cause of negative externalities on social media. Changing the engagement model may conflict with a platform’s business model, so safety proposals can focus on making existing practices less harmful instead. Such changes may reduce harm without addressing its underlying source; overmoderation also risks limiting audiences for historically excluded communities. section 9.3
The limits of harm reduction
The tobacco analogy distinguishes reducing harm from addressing its source: filters may make smoking less damaging, but they do not stop cigarette production. Likewise, making a harmful platform less biased may leave the incentives driving harm in place. section 9.3
