Disproportionate accuracy means that an automated system predicts more accurately for some subgroups than for others. Unequal accuracy can make service quality uneven even when the system improves efficiency overall. section 8.2.2
A support-ticket classifier may be less accurate for new customers and non-English-speaking customers, reducing the quality of service they receive. Similar disparities become higher stakes when automation ranks job candidates or predicts qualification and interview outcomes. section 8.2.2 section 8.2.2
Responsible interpretation requires recognizing that:
- Efficiency gains do not guarantee equal benefits for customers or candidates. section 8.2.2
- Rules-based systems and machine learning models can both produce subgroup-specific inaccuracies. section 8.2.2
- Inaccurate predictions can reinforce bias and affect income, careers, and other long-term outcomes. section 8.2.2
Figure 8.5 The categorization system developed by the data science and analytics team can streamline the work of the customer support team. Still, it may have an unanticipated negative effect on certain groups of customers.
Use the figure to connect streamlined support work with its possible negative effects on particular customer groups. section 8.2.2
