liveBook · Manning

Construct validity

As explained in Hidden Influences

The extent to which measured indicators adequately represent an abstract construct that cannot be measured directly.

A recommendation can keep someone scrolling without giving them anything they value. Construct validity asks whether measurable indicators—such as engagement—adequately represent an abstract concept that cannot be measured directly, such as value. section 4.2 section 4.3

Recommender systems infer preferences from interactions and use signals such as dwell time and engagement as proxies for value. But behavior does not always reveal preference: tastes can change, and people may interact with content to argue against it, not because they value it. section 4.2 section 4.3

When engagement misleads

An inflammatory post may attract replies from people correcting or insulting its author. The platform records engagement and may promote the post, even though those interactions do not establish that readers value it. Click-throughs on headlines may likewise reflect curiosity more than satisfaction. section 4.3

Figure 4.1 Recommender systems learn our preferences by observing our interactions with the items they present. There is no guarantee that all the preferences will be appropriately accounted for. In this figure, the user is a book lover, but the system is unaware of their preferences, and it’s reinforcing the known ones.

The figure contrasts a reader’s unobserved interests with the interactions a recommender system can observe, showing how known preferences may be reinforced while others go unaccounted for. section 4.2

How Hidden Influences covers it

How deep it goes
Defines the measurement problem and illustrates proxy failures, without prescribing a general solution.
Examples
Uses inflammatory posts, headline clicks, and recommendation interactions to show how behavior can diverge from value or preference.
What you can do after
Evaluate whether engagement measures recommendation value or may instead reflect other motives.
How it connects
Links construct validity to recommender systems, implicit learning, user preferences, and engagement metrics.

Also explained in

Compare how 2 books explain construct validity →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Hidden Influences. The section links point to the parts of the book it draws on.