A generic dataset accepts arbitrary input without an enforced schema or data validation. The service stores raw data as received and later packs it into a training dataset in its original format. section 2.2.6 section 2.4
This format separates data collection from training-code changes: upstream contributors can introduce new data without synchronously updating a maintained schema. The flexibility is intended for uncertain, experimental projects rather than production systems. section 2.2.6 section 2.2.6
Use a generic dataset when:
- Data formats are still changing and the team needs to test algorithms or collection approaches quickly. section 2.2.6
- The service must accept and return raw data without coordinating every change with downstream training code. section 2.4
- The team can tolerate downstream parsing failures caused by unexpected input. section 2.2.6
The operational cost is deferred rather than removed: downstream training code must interpret arbitrary data, and the absence of type checks can break its parsing logic. Once requirements mature, promote the format to a strongly typed dataset. section 2.2.6
Figure 2.15 Each type of dataset has two data schemas: ingestion data schema and training data schema. These two schemas will ensure that the data we accept and the data we produce follow our data spec.
Use the dataset-schema architecture to contrast validated ingestion and training contracts with the generic dataset exception, which omits schema validation. section 2.2.6
