liveBook · Manning

Baseline

As explained in Hidden Influences

A reference condition or measurement representing a neutral or expected state against which system outcomes are compared.

A body temperature of 38°C is easier to interpret when compared with a usual temperature of roughly 37°C. That reference is a baseline: a condition or measurement used to judge how much an outcome differs from what is expected or chosen for comparison. section 4.1

A useful baseline depends on the question. To study safer city roads, 0 km/h would describe a situation in which no car moves; a speed such as 30 km/h may offer a more useful reference. Choosing a baseline can require knowledge of the domain, not just a measuring tool. section 4.1

Different questions can call for different baselines:

  • A handwritten-digit recognizer can be compared with random guessing, which would correctly identify roughly 10% of samples. section 4.5
  • A proposed replacement system can be compared with the system already in use. Accuracy may still need to be weighed against running cost or speed. section 4.5
  • A social media feed can be compared with a reverse chronological feed, which puts the newest items first. This became a common reference partly because it preceded newer ranking systems. section 4.5

A common reference is not necessarily a neutral or universal one. Recommender systems respond to personal preferences, which change and are themselves shaped by past recommendations. That makes it difficult to establish how much exposure content should receive without the system’s influence. section 4.5 section 4.8

Comparing exposure in an A/B test

A platform can use its existing recommender as the baseline while testing a new one. For a particular piece of content, an exposure ratio of 1 means the systems offer the same exposure; 2 means the new system offers twice as much, and 1/3 means it offers one-third as much. The comparison describes amplification relative to the old system, not an absolute measure of what the content deserves. section 5.5.2

How Hidden Influences covers it

How deep it goes
Defines baseline selection through comparisons and explains why recommender systems lack a universal reference.
Examples
Illustrates baselines with body temperature, road speed, handwritten-digit recognition, and exposure ratios between recommender systems.
What you can do after
Evaluate and understand how a chosen reference affects comparisons of system performance and content exposure.
How it connects
Connects baselines to recommender systems, algorithmic amplification, reverse chronological order, and A/B testing.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Hidden Influences. The section links point to the parts of the book it draws on.