liveBook · Manning

Moderator

As explained in Hidden Influences

A person who reviews user-generated content and decides whether it complies with a platform's rules.

When a post is reported or flagged for review, a human moderator considers whether it violates a platform’s rules. Moderators apply those rules to individual cases, where context can make an apparently simple decision difficult. section 3.4 section 3.2

A moderator’s work sits within a larger system:

  • Automated systems may remove content or flag uncertain cases for human review; user reports can also send content to a review queue. section 3.4
  • Reviewers may decide to remove content or sanction a user. High volumes and pressure to work quickly can leave little time to consider context. section 3.4 section 3.5.2
  • Repeated exposure to disturbing material can cause serious psychological strain. Outsourcing and performance pressure can compound the difficulty of the work. section 3.5.2

Why context matters

A rule against nudity can require exceptions: art may be allowed, as may breastfeeding. A historical image can also prompt a policy change. Moderators must apply definitions while handling cases whose meaning depends on context. section 3.2

Figure 3.2 An example of a moderation workflow. When a new piece of content is created (top left), it might be scanned by algorithmic moderators that can remove it or flag it for further human review if a decision can’t be made. Content that is reported from other users goes into the same queue. Human moderators review content from this ever-growing queue, making a final determination, including a user’s ban if they have been a repeat offender, or if the posted content warrants such a decision.

The workflow shows how automated checks and user reports can lead to a shared queue for human decisions. section 3.4

How Hidden Influences covers it

How deep it goes
Applies company definitions to individual cases, while acknowledging cultural disagreement, context, and errors at scale.
Examples
Uses nudity exceptions, including art, breastfeeding, and historically significant images, to illustrate contextual decisions.
What you can do after
Understand how moderators make case-by-case enforcement decisions and how exposure and workload affect their work.
How it connects
Connects content moderation with terms of service, user reporting, and algorithmic moderation.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Hidden Influences. The section links point to the parts of the book it draws on.