Error rate by subgroup measures incorrect model outputs separately for meaningful user, item, or scenario slices. It can expose quality gaps hidden by aggregate performance. section 3.2.1
Define the error before calculating the metric. Classification errors may be false positives or false negatives; recommendation errors may be irrelevant recommendations, missed relevant items, or relevant items ranked too low. section 3.2.1
For recommendation systems, useful error-style measures include:
- Irrelevant recommendation rate: the share of recommended items that are not relevant, equivalent to the inverse of precision@K. section 3.2.1
- Miss rate: the share of relevant items that the model failed to recommend, equivalent to the inverse of recall@K. section 3.2.1
Preferred-genre recommendation slices
Group users by their most-watched genre, then compare recommendation errors across action, drama, comedy, and indie audiences. A much higher indie miss rate indicates that this subgroup receives worse coverage and warrants investigation. section 3.2.1
