A training dataset is the collection of examples a model learns from. Its value depends on more than size: the examples must reflect the people, questions, and situations the product will encounter. A financial-data assistant trained on question-to-SQL pairs can perform well on test data yet fail with real users if those pairs do not match the questions users actually ask. section 1.3.2 section 1.1
For supervised learning, each input needs a target label that supplies a learning signal. A user classifier, for instance, can learn from customer records paired with segment labels previously assigned during clustering. Other datasets may begin with proxy labels, automatically generated labels, product feedback, or human annotation. Each route needs scrutiny: movie reviewers may skew toward extreme ratings, and human labeling needs clear guidelines to limit inconsistency and subjective bias. section 4.2.1
Table 4.2 A sample of labeled examples
| Features |
Label |
|||||
| user_id |
purchased_items |
purchase_value |
last_active |
n_visits |
search_queries |
segment |
| abhj3k |
2 |
908 |
2024-04-30 08:36:24 |
48 |
3 |
Seekers |
| shj67d |
0 |
0 |
2023-12-26 12:56:24 |
24 |
45 |
Conservatives |
| i963gh |
12 |
673 |
2024-05-15 23:22:11 |
156 |
25 |
Indecisives |
| ty54df |
20 |
1250 |
2024-05-10 14:21:07 |
190 |
5 |
Champions |
The table shows customer features beside their segment labels, making the input–target pairing concrete. section 4.2.1
Before relying on a dataset, check what it leaves out and what it may carry forward:
- Compare examples with actual user requests; build a separate test suite from real-life questions rather than trusting test performance alone. section 1.1
- For a third-party language model, examine the scope of its training data. Missing languages, newer slang, or specialized content can limit otherwise fluent output. section 5.1.1
- Audit representation before training, add data for underrepresented groups where appropriate, and monitor for distribution changes after deployment. section 11.3.1
When historical data becomes a hiring rule
A hiring model trained on past applicants may favor an overrepresented educational background rather than equally qualified candidates from other backgrounds. Historical hiring decisions can also encode discrimination: Amazon’s reported hiring-tool case illustrates the risk of reproducing past bias. Dataset review is therefore a product and governance decision, not just a model-tuning task. section 11.3.1 section 11.3.4
