A REST API can make a trained model available online for inference. Exporting its weights and computation graph lets it be served outside Python, including from environments such as C++. section 6.3.2
This option suits applications with reliable internet access and no strict latency requirements; a request, inference, and response round trip typically takes around 500 ms. Mobile apps may be unavailable in airplane mode or low-connectivity settings. section 6.3.2
model.export("path/to/location", format="tf_saved_model") #1 reloaded_artifact = tf.saved_model.load("path/to/location") #2 predictions = reloaded_artifact.serve(input_data) #2
See the Keras export, reload, and serving flow for a TensorFlow SavedModel artifact. section 6.3.2
model.export("path/to/location", format="onnx") ort_session = onnxruntime.InferenceSession("path/to/location") predictions = ort_session.run(None, input_data)
See the corresponding Keras export and inference flow using ONNX Runtime. section 6.3.2
