Dataset versioning preserves the identity and provenance of training data so teams can reproduce fine-tuning runs, investigate model regressions, and support audit trails. Pair each dataset version with the training run and resulting model checkpoint. section 3.5
A minimum viable versioning system should:
- Content-hash each dataset at creation and store the hash in a manifest that travels with the data. section 3.5
- Log each run’s dataset SHA-256, full hyperparameter configuration, and per-category evaluation scores. section 3.5
- Record each example’s source, annotation, and transformation provenance as it enters the pipeline. section 3.5
Trace a dataset change
When creating v2 by adding targeted synthetic examples to v1, put v1’s SHA-256 in v2’s manifest as its parent. If a model regresses, compare hashes: matching hashes point the investigation toward hyperparameters or infrastructure; differing hashes let the team inspect added, removed, or changed examples. Provenance can further distinguish a source or annotation quality problem from reduced category coverage. section 3.5.3 section 3.5.3 section 3.5
Experiment trackers such as Weights & Biases and MLflow can link hyperparameters, infrastructure, loss curves, and evaluation scores to dataset and checkpoint identifiers. section 3.5
