liveBook · Manning

TFLiteConverter

As explained in Quantization and Fast Inference

TFLiteConverter is the TensorFlow Lite API for converting models to TensorFlow Lite format.

TFLiteConverter converts TensorFlow models to .tflite artifacts and applies post-training quantization in one pass, including graph optimization, operator fusion, calibration, and serialization. Its compact workflow offers less layer-level control than PyTorch PT2E. section 6.3

Choose a conversion path by the required weight and activation types:

  • Dynamic range uses INT8 weights without calibration; activations remain FP32. section 6.3
  • Full integer uses a representative dataset to calibrate INT8 weights and activations. section 6.3
  • Float16 stores weights as FP16. section 6.3

Conversion settings do not determine deployment behavior alone: the runtime delegate and target device also matter. The API surface remains unchanged under the LiteRT runtime name, and the file format remains .tflite. section 11.2

Listing 6.4 TFLite converter configurations
import tensorflow as tf
converter = tf.lite.TFLiteConverter.from_saved_model(saved_model_path)
 
# Mode 1: Dynamic range — weights to INT8, no calibration              #A
converter.optimizations = [tf.lite.Optimize.DEFAULT]
 
# Mode 2: Full integer — weights AND activations to INT8               #B
converter.optimizations = [tf.lite.Optimize.DEFAULT]
converter.representative_dataset = representative_dataset_fn
 
# Mode 3: Float16 — weights stored as FP16                             #C
converter.optimizations = [tf.lite.Optimize.DEFAULT]
converter.target_spec.supported_types = [tf.float16]
 
tflite_model = converter.convert()

Configuration listing for dynamic-range, full-integer, and float16 conversion. section 6.3

How Quantization and Fast Inference covers it

How deep it goes
Explains three configuration paths and single-pass conversion, but not selective layer quantization.
Examples
Uses MobileNetV2 and BERT-base to compare conversion configurations and quantization results.
What it assumes
Assumes a TensorFlow SavedModel and a representative dataset for full-integer calibration.
What you can do after
Apply it by configuring a quantization path, supplying calibration data when needed, and converting the model.
How it connects
Connects post-training quantization and calibration to runtime delegates and PyTorch PT2E comparisons.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Quantization and Fast Inference. The section links point to the parts of the book it draws on.