Fairness is not guaranteed by a base model: even clean data can yield bias that grows during fine-tuning. Treat equitable behavior as something to enforce, not assume. section 11.3
Unequal responses across names
A model gives shorter responses to names like Jamal and Maria than to John and Emily. A demographic-gap penalty can discourage different treatment during training, though stronger penalties may slightly reduce accuracy. section 11.3.2
Address fairness through training and ongoing evaluation:
- Add a penalty comparing prediction distributions across demographic groups to the usual accuracy loss. section 11.3.2
- Measure performance separately across groups; use demographic parity or equalized odds, diverse test examples, and training-data audits to find disparities and trace their causes. section 11.6.6
import torch.nn.functional as F
def fairness_loss(model_output, labels, groups, fairness_weight=5.0):
base_loss = F.cross_entropy(model_output, labels)
group0 = model_output[groups == 0]
group1 = model_output[groups == 1]
gap = (group0.softmax(dim=-1).mean() -
group1.softmax(dim=-1).mean()).abs().mean()
return base_loss + fairness_weight * gap
This PyTorch loss sketch adds a weighted difference between groups’ average softmax predictions to cross-entropy. Raising the weight strengthens the fairness penalty and can trade off against accuracy. section 11.3.2
In production, compare outcomes across user segments and review interactions regularly; averages and automated metrics can miss nuanced failures. Fairness is part of reliability, alongside accuracy and privacy. section 11.6.6 section 1.4.3
