Flexibility keeps an inference system ready for changing requirements. A durable design can absorb a new model, framework, preprocessing or postprocessing step, feature, or API without disrupting existing functionality. section 15.1
Design for flexibility by checking whether the system can:
- Adopt models trained with different frameworks, including through conversion before inference. section 15.1 section 15.6
- Add preprocessing, postprocessing, features, or APIs without affecting current behavior. section 15.1
- Balance experimental freedom against production performance, using monitoring and integration tests when environments diverge. section 15.2.1
Framework changes without rebuilding the pipeline
A model can be trained in one framework and converted for inference in another, allowing a later framework switch with new features without changing the inference pipeline. section 15.6
