A dataset collects data instances and may pair them with labels for supervised learning. In an OCR system, it supplies the examples used to detect ID cards and fine-tune an image detector. section 1.4.2
Production workflows treat datasets as operational inputs rather than static files: teams build them, train models with them, deploy services, and monitor new inference data against historical data. section 1.4.2 section 6.2.1
Use datasets to support distinct stages of an ML system:
- Train and fine-tune models with labeled examples, such as ID-card images for an image detector. section 1.4.2
- Compare historical reference data with current inference data to detect changes in feature distributions. section 6.2.1
- Measure dataset-level and column-level properties, including missing values, descriptive statistics, and drift scores. section 6.2.1
Reference and current datasets
A drift report receives a historical reference dataset and the current data targeted for prediction. Evidently compares corresponding feature distributions, records the selected statistical test and drift score, and can render the result as an HTML report. section 6.2.1
Listing 6.5 Generating a data drift report using DataDriftPreset
report = Report(metrics=[
DataDriftPreset(),
]) #1
report.run(
reference_data=reference_data,
current_data=current_data
) #2
report.save_html("drift_report.html") #3
The runnable report pattern shows how to pass reference_data and current_data to DataDriftPreset and save the resulting report. section 6.2.1
