liveBook · Manning

Amplification metric

As explained in Hidden Influences

A quantitative measure of the increase in content visibility, reach, or influence produced by a distribution process.

A social media feed can make some posts more visible than others. An amplification metric measures an aspect of that extra exposure, helping distinguish what a recommender system does from what was already visible to users. section 5.1

No single measurement gives a complete picture. Just as temperature alone cannot describe the weather, different metrics answer different questions about a recommender. Useful comparisons should account for differences between users, including how often they post and how many followers they have. section 5.1

Table 5.1 Types of algorithmic amplification
Metric
Side
Question being answered
A/B tests
Author
How much extra exposure does specific content receive, thanks to the algorithmic recommendations?
Normalized impressions
Author
How can we compare different kinds of users?
Out-of-network ratio
Author
How much exposure comes from direct followers rather than from showing the content to other users?
Feed analysis
Reader
Do the recommendations match the intent of the reader?

The table compares four approaches: A/B tests, normalized impressions, out-of-network ratio, and feed analysis. The first three ask questions from the author’s side; feed analysis asks whether recommendations match the reader’s intent. section 5.5

An impression is one view of a post, not necessarily one distinct viewer: the same person may see a post more than once. A measurement also needs a time frame, because a post can accumulate more impressions over a week than over a day. section 5.5

Comparing positions in a policy debate

To ask whether a feed promotes supporters of a proposed policy more than opponents, researchers must identify and label posts on each side, then compare their relative amplification. Finding relevant posts at scale raises questions of privacy and nuance. section 4.7

Choosing what to measure is also a question of power. If an impact such as misinformation is not measured, it can be left out of the reported picture. Independent researchers and stakeholders with different perspectives can help define and validate metrics, though too many metrics can make practical decisions harder. section 5.3

How Hidden Influences covers it

How deep it goes
Develops desirable measurement characteristics and operational definitions, with mathematical definitions and high-level explanations of their questions.
Examples
Uses weather measurement, varied social media users, and a policy debate to illustrate comparison and labeling challenges.
What you can do after
Evaluate recommender exposure from producer and consumer perspectives while considering time frames and labeling limits.
How it connects
Connects amplification to recommender systems, impressions, A/B tests, and feed analysis.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Hidden Influences. The section links point to the parts of the book it draws on.