liveBook · Manning

Fairness pruning

As explained in Rearchitecting LLMs

Fairness pruning is a model-modification technique that removes selected neurons to change unwanted behavioral tendencies while preserving performance on desired capabilities.

Fairness pruning modifies model behavior by targeting neurons associated with demographic attributes, aiming to reduce unwanted tendencies without harming desired capabilities. The intervention can be graded: scaling neurons’ contributions reduces them, while zeroing them removes them. section 1.4 section 11.5.2

A neuron-targeting workflow links behavioral signals to an intervention:

  • Compare activations for contrastive prompt pairs at the input to the MLP down_proj layer to identify candidate neurons. section 11.5.2
  • Rank candidates by BiasScore and scale their up_proj contributions to reduce their influence rather than eliminate them. section 11.5.2

Across evaluated models, languages, and categories, the strongest signal was in the final layer; gender was less concentrated. Candidate neurons showed limited overlap between categories, and interventions measurably changed processing of the associated attribute. section 11.5.2

Figure 11.8 Jaccard overlap between the five demographic categories evaluated in Llama-3.2-1B in English, over the top 0.1% neurons by BiasScore per category, 131 neurons per category. Values below 0.33 indicate that between roughly 51% and 72% of each category's candidate neurons are exclusive to it. Figure reproduced from Martra et al. (2026).

The figure compares candidate-neuron overlap across five demographic categories, illustrating their degree of specialization. section 11.5.2

How Rearchitecting LLMs covers it

How deep it goes
Explains neuron localization, category overlap, causal intervention, and graded scaling, but gives no specific code details here.
Examples
Compares five demographic categories in Llama-3.2-1B and evaluates models in English and Spanish.
What you can do after
Apply fairness pruning by comparing contrastive-prompt activations, selecting high-BiasScore neurons, and scaling their up_proj contributions.
How it connects
Connects activation analysis and MLP neurons with BiasScore, contrastive prompts, and Jaccard overlap.

This explainer was generated by Manning's knowledge graph from the text of Rearchitecting LLMs. The section links point to the parts of the book it draws on.