TFLite is a framework for efficient on-device deep-learning inference. It targets Android and iOS smartphones, ARM CPUs, Raspberry Pi, and certain microcontrollers, keeping sensitive inputs local and avoiding network latency. section 6.3.2
Choose on-device inference when:
- Sensitive data cannot be sent to a remote model, such as end-to-end encrypted chat messages. section 6.3.2
- Latency matters enough to avoid a network round trip, or the application must continue working without connectivity. section 6.3.2
- Export a Keras model with its computation graph, then deploy the resulting TensorFlow-compatible artifact on the device. section 6.3.2
model.export("path/to/location", format="tf_saved_model") #1 reloaded_artifact = tf.saved_model.load("path/to/location") #2 predictions = reloaded_artifact.serve(input_data) #2
Use the export-and-reload workflow to produce a deployable TensorFlow SavedModel before serving predictions locally or integrating the artifact with an on-device runtime. section 6.3.2
