liveBook · Manning

Exploration-exploitation trade-off

As explained in Hidden Influences

The decision problem of balancing recommendations known to perform well against unfamiliar recommendations that may reveal new user interests.

Suppose a recommender learns that you respond to dog videos. It can keep showing familiar successes (exploitation) or try unfamiliar items (exploration) to uncover interests that have not surfaced. section 4.2

Familiar picks can deliver short-term value yet miss broader interests; distant experiments can feel irrelevant and push users away. The balance depends on each user’s tolerance for discovery. section 4.2

Because preferences are inferred from interactions, observed behavior may not capture every interest; treating it as a complete map raises a construct-validity question. section 4.2

Figure 4.1 Recommender systems learn our preferences by observing our interactions with the items they present. There is no guarantee that all the preferences will be appropriately accounted for. In this figure, the user is a book lover, but the system is unaware of their preferences, and it’s reinforcing the known ones.

The illustration shows a recommender reinforcing known interests while missing the user’s love of books. section 4.2

How Hidden Influences covers it

How deep it goes
Explains short-term benefits, risks of excessive exploration, and personalization without detailing an optimization method.
Examples
Uses dog-content preferences and a book-lover illustration to show how known interests can crowd out undiscovered ones.
What you can do after
Evaluate how recommendation choices trade immediate relevance against discovery and user tolerance.
How it connects
Connects recommender systems and implicit learning to user preferences and construct validity.

Also explained in

Compare how 6 books explain exploration-exploitation trade-off →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Hidden Influences. The section links point to the parts of the book it draws on.