Model comparison evaluates training experiments or resulting models by retaining the metadata and artifacts needed to inspect performance, reproduce results, and select a strong candidate. section 8.0
A comparison workflow depends on a persistent metadata and artifact store that can:
- Connect model versions to their training runs, input parameters, datasets, code versions, and training metrics through lineage. section 8.3.1
- Retain pipeline configuration, input datasets, algorithm code versions, and artifacts so a model can be reproduced for evaluation or troubleshooting. section 8.3.1
- Provide manual and programmatic access to packaged model files for debugging and automated performance testing. section 8.3.1
Investigating a degraded production model
Start with the deployed model version, retrieve its lineage and model file, then inspect the training dataset, code version, parameters, and metrics to identify a likely cause of performance degradation or compare it with another run. section 8.3.1
