The Netflix Prize abstracted the recommendation problem to a simplified proxy of accurately predicting ratings. It is now clear that this is just one of many components in an effective industrial recommendation system. They also need to account for factors like diversity, context, evidence, freshness, and novelty.
Xavier Amatriain et al.[1]
1 Amatriain, Xavier et al., Past, Present, and Future of Recommender Systems: An Industry Perspective (Recsys, 2016).
After studying this chapter, you’ll gain experience in the following areas:
- Evaluating the effectiveness of a recommender algorithm
- Splitting data sets into training data and test data
- Building offline experiments to evaluate recommender systems
- A rough understanding of online testing
Why did you implement a recommender? What did you want to gain? Do you want to earn more? Have more visitors? Try out new technology? No matter what you answer, it might not directly translate into a way to calculate whether or not you’re improving.[2] You often hear about algorithms that are better or slightly improved compared to the current cutting-edge algorithms, but improving what and how?
2 Start with Why: How Great Leaders Inspire Everyone to Take Action (Portfolio; Reprint edition, 2011) by Simon Sinek, is completely unrelated to recommender systems, but is an interesting book on how you need to understand why your business is there.