An ML release cycle resembles regular software delivery, but model changes require slower, harder-to-interpret validation and may need human judgment before deployment. section 13.2
Plan for two release complications:
- Allow for longer validation because retraining a model takes substantially more time than compiling code or building ordinary artifacts. section 13.2
- Assess regressions rather than trusting tests alone; an overall score can improve while harmful errors appear elsewhere. section 13.2
- Assign a human reviewer when tradeoffs require contextual judgment, using an ML engineer, product manager, domain expert, or aggregated user feedback. section 13.2
- Consider automatic model release in simpler cases where AutoML supports the model decision without automating the whole system. section 13.2
A small score improvement can still be a bad release
A new model corrects three previously mislabeled test samples but introduces two new errors among 100 samples. The aggregate result improves, yet release approval still depends on whether the new errors are acceptable. section 13.2
Figure 13.3 Blue-green deployment
Inspect blue-green deployment as a release rollout pattern. section 13.2
Figure 13.4 Canary deployment
Inspect canary deployment as a release rollout pattern. section 13.2
