Have you ever found it difficult to rate something (food, a product, or an experience) on an exact scale? Asking for the customer’s numerical score for a product is a common task in A/B testing and product recommendation workflows.
Definition
A/B testers and product recommendation engineers often have to deal with a high level of noise in the feedback collected from their customers. By noise, we mean any type of corruption that the feedback collected from customers is subject to. Example sources of noise in product rating include the number of advertisements served on an online streaming service, the quality of the delivery service for a package, or the general mood of the customer when they consume a product. These factors affect how the customer rates their product, potentially corrupting the signal that is the customer’s true evaluation of the product.