4 Bias and fairness: Modeling recidivism

In our last chapter, we focused on building a feature engineering pipeline that would maximize our model’s performance on our dataset. This is generally the stated goal of an ML problem. Our goal in this chapter will be to not only monitor and measure model performance but also to keep track of how our model treats different groups of data because sometimes data are people.

In our case study today, data are people whose lives are on the line. Data are people who simply want to have the best life they can possibly have. As we navigate the waters around bias and discrimination, around systemic privilege and racial discrepancies, we urge you to keep in mind that when we are talking about rows of data, we are talking about people, and when we are talking about features, we are talking about aggregating years—if not decades—of life experiences into a single number, class, or Boolean. We must be respectful of our data and of the people our data represents. Let’s get started.

4.1 The COMPAS dataset

4.1.1 The problem statement and defining success

4.2 Exploratory data analysis

4.3 Measuring bias and fairness

4.3.1 Disparate treatment vs. disparate impact

4.3.2 Definitions of fairness

4.4 Building a baseline model

4.4.1 Feature construction

4.4.2 Building our baseline pipeline

4.4.3 Measuring bias in our baseline model

4.5 Mitigating bias

4.5.1 Preprocessing

Summary