An automated decision tool is software that applies fixed rules or learned algorithms to make or support decisions without a person determining every case. It can rank, categorize, score, select, or reject people and cases, making technical design choices consequential for organizational strategy and individual outcomes. section 8.2.2
Common forms include:
- Machine-learning software that ranks job candidates. section 8.2.2
- Internally developed keyword rules that match candidate skills to job requirements. section 8.2.2
- Vendor systems that generate candidate quality scores and automatically reject people below a threshold. section 8.2.2
Automation does not remove the possibility of harm: rules and models can produce different scores or selection rates across protected groups such as race, ethnicity, sex, or intersecting groups. Responsible use therefore requires auditing predicted rankings, ratings, and selection rates, while considering privacy, consent, and data minimization when personal information is involved. section 8.2.2 section 8.4
Customer-support categorization
A categorization system can streamline customer-support work while producing an unanticipated negative effect on certain customer groups. The example illustrates why operational efficiency should be evaluated alongside subgroup outcomes. section 8.2.2
Figure 8.4 An automated system for categorizing data can employ anything from simple rules to sophisticated machine learning algorithms.
Compare simple rules with sophisticated machine-learning algorithms as alternative ways to automate categorization. section 8.2.2
