11 Precision pruning for bias

The training process of large language models relies heavily on text and information freely available on the internet, which may or may not represent the values and realities of a given society, or may even lead to responses we don't want to see in our specific environment.

That kind of deviation, when the model treats an identical case differently just because a demographic attribute like race or gender changes, is what we call bias. In this chapter, you're going to learn a technique to localize that specific behavior inside the model and soften it, by changing the weight of just a few neurons, without needing to retrain the model.

If you're in disbelief, don't worry; that's a normal reaction. It's the look I've seen on most people's faces when I explain this process to them. Disbelief tends to turn into curiosity once they understand what the modification consists of and what they can achieve with it.

11.1 Comparing activations between prompts

11.1.1 Creating and verifying the prompt pairs

11.1.2 Where the differences show up

11.2 Finding and scaling down the neurons behind racial bias

11.2.1 Measuring the activation difference

11.2.2 Visualizing the differences across layers

11.2.3 Selecting the top candidate neurons

11.2.4 Intervening on the top-k neurons

11.2.5 What changed after scaling

11.2.6 Same bias, different scenario

11.3 Replicating the procedure with gender bias

11.3.1 Locating the gender neurons

11.3.2 Not every neuron has the same influence

11.3.3 Tracing the effect across layers

11.4 Benchmarking the intervened models

11.4.1 Measuring the impact on general capabilities

11.4.2 Measuring bias with BBQ

11.5 From paper to practice

11.5.1 Anthropic, the starting point

11.5.2 Fairness Pruning, one step ahead

11.5.3 Hands-on lab

11.6 Summary