A framework provides reusable infrastructure for configuring and executing machine-learning work. Training and inference often favor different priorities: research needs flexibility and simplicity, while production serving needs performance, stability, and long-term maintainability. section 15.3.1
Choose a framework strategy by weighing these practical options:
- Use separate training and inference frameworks when research tools such as Pandas, scikit-learn, or Keras would constrain production serving. section 15.3.1
- Decouple the two pipelines so a future training-framework change does not disrupt an inference system expected to operate for years. section 15.3.1
- Use a unified approach when appropriate; Torch 2.0 compilation can produce an optimized inference pipeline from training and experiment code. section 15.3.1
- Use ONNX as an interframework format when conversion, portability, runtime support, and optimization matter. section 15.3.1
A production-oriented split
A team can train with a flexible research framework, convert the model to ONNX, and serve it through ONNX Runtime with backend-specific optimizations. section 15.3.1
Figure 15.3 A wide spectrum of tools with their focus on either research or production serving goals
Use the spectrum to compare tools by their emphasis on research flexibility or production serving. section 15.3.1
