Data curation
Data curation is the systematic selection, review, cleaning, organization, and maintenance of data for a particular purpose.
Introduction to Generative AI, Second Edition
Dataset curation is the process of selecting, organizing, documenting, reviewing, maintaining, and quality-controlling data for a particular purpose.
- How it is used
- Positions data curation as part of bias mitigation and the legal and ethical governance of generative AI.
- How deep it goes
- Identifies documentation, balancing, and review as safeguards but does not provide a step-by-step curation procedure.
- Examples
- Uses Hugging Face dataset cards, the Data Nutrition Project’s labels, and a card for databricks-dolly-15k as documentation examples.
- What you can do after
- Evaluate/understand it by checking a dataset’s contents, demographic coverage, limitations, and intended use.
Read how Introduction to Generative AI, Second Edition explains data curation →
Deep Learning with R, Third Edition
Data curation is the selection, cleaning, organization, and quality management of data for analysis or machine learning.
- How it is used
- Improves deep-learning generalization by making training data informative, low-noise, and densely representative.
- How deep it goes
- Connects curation to manifold coverage and interpolation, but stops short of a procedural curation workflow.
- Examples
- Uses MNIST with randomly shuffled labels to expose the consequences of corrupted training relationships.
- What you can do after
- Apply via data improvements that increase informative, accurate, low-noise, and dense coverage before changing the model.
Read how Deep Learning with R, Third Edition explains data curation →
Related concepts
- rejection samplingincludes · Rejection sampling is a method that generates candidate samples and accepts those meeting a specified criterion while discarding the others.
- pipelinepart of · A pipeline is an ordered or coordinated sequence of connected processing stages or tasks that passes data, work, or results toward an outcome.
- model architecturecontrasts with · The structural organization of a computational system’s components, relationships, and operations that determines how it processes information or behaves.
- datasetuses · A dataset is an organized collection of related data items, records, or observations treated as a unit for storage, processing, analysis, or computation.
- fine-tuninguses · Fine-tuning is adapting a pretrained machine-learning model by training it further on targeted data or objectives for a task, domain, or behavior.
- lifecyclepart of · A lifecycle is the sequence of stages or states an entity, process, or artifact passes through during its existence.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.