State persistence keeps datasets available across ingestion, storage, and training workflows. For training systems, it supports reproducibility, troubleshooting, and consistent retrieval as data grows from megabytes to petabytes. section 2.1.2 section 2.4
A practical design separates data ingestion, internal dataset storage, and training-data fetching. The APIs remain consistent across dataset types and sizes while storage handles scale internally, such as through cloud object storage. section 2.2.8 section 2.4
Implement persistence with explicit lifecycle guarantees:
- Store every dataset update as an immutable commit, preserving the history needed to reproduce changes. section 2.2.8
- Store every prepared training dataset as an immutable, versioned snapshot with no data-expiration limit. section 2.2.8
- Fetch training data through a version hash so the same dataset can be retrieved for later training or investigation. section 2.4
- Prefer soft deletion for ordinary removal, reserving hard deletion for requirements such as permanent customer-data erasure. section 2.1.2
Versioned training workflow
An update request creates a new dataset commit. A subsequent training-data preparation request creates a versioned snapshot. Training code records the snapshot’s version hash, allowing the exact input data to be fetched later for reproducibility or performance troubleshooting. section 2.2.8 section 2.4
Persistence is a system property, not merely a storage choice: schemas, APIs, version tracking, deletion behavior, and audit operations determine whether retained data remains usable and trustworthy. section 2.2.8 section 2.4
