Production is the environment where a deployed model serves an application. Unlike training, production use focuses on inference: generating predictions. The deployed model may be exported from Python and optimized for speed or memory. section 6.3.2
Choose a deployment approach that fits the application:
- Serve predictions through a REST API using TensorFlow Serving or ONNX. This suits applications with reliable internet access; a remote API can leave a mobile app unusable offline or with poor connectivity. section 6.3.2
- Export the model when the production environment does not support Python, or when using Python to serve it would add overhead. Inference-only use also allows speed and memory optimizations. section 6.3.2
model.export("path/to/location", format="tf_saved_model") #1 reloaded_artifact = tf.saved_model.load("path/to/location") #2 predictions = reloaded_artifact.serve(input_data) #2
Export a Keras model as a TensorFlow SavedModel, load the artifact, and use its serving function to generate predictions. section 6.3.2
model.export("path/to/location", format="onnx") ort_session = onnxruntime.InferenceSession("path/to/location") predictions = ort_session.run(None, input_data)
Export a Keras model to ONNX, then load it with ONNX Runtime to generate predictions. section 6.3.2
