A balanced dataset represents its classes or labels in relatively comparable proportions. Balance matters because examples dominated by one class can bias a classifier or language model toward that majority. section 4.2.1 section 6.2.1
Use balance as one part of dataset quality rather than treating it as a substitute for sound data preparation:
- Compare the number of examples across classes and reduce the risk that larger classes dominate predictions. section 4.2.1
- Check label accuracy and consistency, using annotation guidelines and agreement checks when multiple annotators are involved. section 4.2.1
- Preserve diversity and domain relevance so balanced examples still represent messy real-world inputs and the target application. section 4.2.1
- For few-shot prompts, balance the labels among demonstrations to reduce majority-label bias; review example order separately because recency can also affect outputs. section 6.2.1
Sentiment-analysis demonstrations
If most few-shot examples are positive, a language model may classify new inputs as positive too often. Adding a comparable set of negative and other relevant labels can reduce this majority-label bias, although example recency and common-token bias still require attention. section 6.2.1
