A social media feed can make some posts more visible than others. An amplification metric measures an aspect of that extra exposure, helping distinguish what a recommender system does from what was already visible to users. section 5.1
No single measurement gives a complete picture. Just as temperature alone cannot describe the weather, different metrics answer different questions about a recommender. Useful comparisons should account for differences between users, including how often they post and how many followers they have. section 5.1
Table 5.1 Types of algorithmic amplification
|
Metric
|
Side
|
Question being answered
|
|
A/B tests
|
Author
|
How much extra exposure does specific content receive, thanks to the algorithmic recommendations?
|
|
Normalized impressions
|
Author
|
How can we compare different kinds of users?
|
|
Out-of-network ratio
|
Author
|
How much exposure comes from direct followers rather than from showing the content to other users?
|
|
Feed analysis
|
Reader
|
Do the recommendations match the intent of the reader?
|
The table compares four approaches: A/B tests, normalized impressions, out-of-network ratio, and feed analysis. The first three ask questions from the author’s side; feed analysis asks whether recommendations match the reader’s intent. section 5.5
An impression is one view of a post, not necessarily one distinct viewer: the same person may see a post more than once. A measurement also needs a time frame, because a post can accumulate more impressions over a week than over a day. section 5.5
Comparing positions in a policy debate
To ask whether a feed promotes supporters of a proposed policy more than opponents, researchers must identify and label posts on each side, then compare their relative amplification. Finding relevant posts at scale raises questions of privacy and nuance. section 4.7
Choosing what to measure is also a question of power. If an impact such as misinformation is not measured, it can be left out of the reported picture. Independent researchers and stakeholders with different perspectives can help define and validate metrics, though too many metrics can make practical decisions harder. section 5.3
