Offline evaluation estimates how a recommender would have performed by training on older interactions and checking predictions against a later holdout period. It is a fast, low-cost proxy—not a guarantee of production performance. section 4.2.3 section 4.1.1
A typical evaluation workflow:
- Prepare interaction logs, then split them by time into older training data and recent test data. section 4.2.1
- Train a model, generate recommendation lists for test users, and compare the lists with their actual interactions using metrics. section 4.2.1
- Compare results with baselines, such as popularity recommendations; use the comparison to decide whether to test online, iterate, or abandon the approach. section 4.2.1
Historical logs reflect the previous system’s exposure and ranking choices, while users may behave differently when shown new recommendations. Treat offline scores as evidence for selecting candidates, not proof of user impact. section 4.1.1 section 4.2.3
Figure 4.8 The offline evaluation workflow. Starting with historical data, we split it into temporal segments, train models, generate recommendations, calculate metrics, and compare them against baselines to decide whether to proceed to online testing.

The workflow diagram summarizes the path from historical data through temporal splitting and model comparison to an online-testing decision. section 4.2.1




