Fairness pruning modifies model behavior by targeting neurons associated with demographic attributes, aiming to reduce unwanted tendencies without harming desired capabilities. The intervention can be graded: scaling neurons’ contributions reduces them, while zeroing them removes them. section 1.4 section 11.5.2
A neuron-targeting workflow links behavioral signals to an intervention:
- Compare activations for contrastive prompt pairs at the input to the MLP down_proj layer to identify candidate neurons. section 11.5.2
- Rank candidates by BiasScore and scale their up_proj contributions to reduce their influence rather than eliminate them. section 11.5.2
Across evaluated models, languages, and categories, the strongest signal was in the final layer; gender was less concentrated. Candidate neurons showed limited overlap between categories, and interventions measurably changed processing of the associated attribute. section 11.5.2
Figure 11.8 Jaccard overlap between the five demographic categories evaluated in Llama-3.2-1B in English, over the top 0.1% neurons by BiasScore per category, 131 neurons per category. Values below 0.33 indicate that between roughly 51% and 72% of each category's candidate neurons are exclusive to it. Figure reproduced from Martra et al. (2026).
The figure compares candidate-neuron overlap across five demographic categories, illustrating their degree of specialization. section 11.5.2
