An algorithm audit lets external evaluators independently test platform claims about algorithmic behavior or effects, rather than relying on the platform’s account. section 10.3.2
Assessment may require access to nonpublic data, such as engagement or reach. Vetted researchers can use such access to study systemic risks, while privacy and intellectual-property concerns remain. section 10.3.2
Independent checks have familiar analogies:
- Vehicle crash tests provide consumers with an independent guide to safety. section 10.3.2
- Financial audits let third parties verify company claims using sensitive proprietary information, though conflicts or fraud can undermine the process. section 10.3.2
Figure 10.2 By counting only how many times content from group A is ranked above content from group B, we can see that overall B is ranked higher more times than A. But, A is ranked higher in the most valuable real estate, the top of the list, where users pay the most attention. Users might not even scroll down the full list, making B’s advantage only real on paper.

A ranking assessment can mislead if it counts wins without considering position: appearing near the top may matter more than winning overall. section 10.3.2
Figure 10.3 While a threshold like 5 or 10 might seem a nice round number, and maybe even a natural choice, it’s really just an arbitrary one. By varying the threshold, the result changes.

Results can change when an evaluator varies a threshold, so a seemingly natural cutoff may shape the conclusion. section 10.3.2
