If a recommendation system had shown someone different content, how might they have responded? That is a counterfactual question: it asks what would have happened under an alternative condition that was never observed. section 4.4
Past interactions cannot answer this fully. People generally interact only with items they were shown, so historical data cannot reveal whether they would have clicked unseen content, spent more time on a platform, or encountered a new interest. Comparing a new recommender’s predictions with past behavior offers only partial evidence and reflects what the previous model chose to show. section 4.4
Figure 4.3 Generally, users interact only with the items presented to them. Very likely, there were many other items that users would have interacted with if they had been shown to them. Because the algorithm is learning via past interactions, it’s possible that certain topics (like the Books topic here) were never presented to the users, thus completely missing one of their interests.
The figure illustrates how items never shown to users can conceal interests and leave possible interactions unobserved. section 4.4
