liveBook · Manning

Int16x8

As explained in Quantization and Fast Inference

A quantization configuration that uses 16-bit integer activations and 8-bit integer weights.

INT16×8 pairs INT8 weights with INT16 activations in an experimental TFLite path, intended to recover accuracy when full-INT8 calibration is fragile. section 11.2

On EfficientNet-Lite0, it produced a 5.40 MB model with 50.6% top-1 accuracy. Synthetic random calibration caused a large accuracy drop; a proper ImageNet calibration shard recovers the gap. section 11.2.1

Inspect the converted flatbuffer to verify the actual tensor types:

  • The INT16×8 graph contains 62 INT16, 50 INT8, 50 INT64, 4 INT32, and two FP32 tensors; the FP32 tensors are at the input and output. section 11.2.1
  • Use `tf.lite.Interpreter.get_tensor_details()` to enumerate tensor dtypes rather than relying on the converter’s exit status. section 11.2.1
Figure 11.5 EfficientNet-Lite0 size × ImageNet top-1 accuracy across the four TFLite converter paths, measured on a 1,000-image ImageNet val subset. FP32 at 18.55 MB lands at 66.8% top-1 (87.7% top-5), the unquantized baseline. Dynamic-range at 4.88 MB lands at 66.5% top-1 (87.9% top-5). That is a 3.8× file-size reduction with essentially no accuracy loss. INT8 at 5.14 MB lands at 49.0% top-1 (86.8% top-5); INT16×8 at 5.40 MB lands at 50.6% top-1 (87.4% top-5). The 17-percentage-point top-1 cliff on the two activation-quantized paths is a calibration artifact, not a property of the integer kernels. The command ch11_2_tflite.py convert ran with a synthetic uniform-random representative dataset because data/calib/ was empty at conversion time. The top-5 numbers (within one percentage point of FP32 on all four variants) confirm the cliff is calibration-shape, not capacity loss; a proper ImageNet calibration shard recovers the gap.

Compare model size and top-1 accuracy across converter paths; the top-5 results help distinguish calibration error from loss of model capacity. section 11.2.1

How Quantization and Fast Inference covers it

How deep it goes
Defines the weight and activation dtypes and reports tensor maps and accuracy, but does not detail quantization arithmetic.
Examples
Measures EfficientNet-Lite0 with synthetic random calibration and reports size, top-1, and top-5 results.
What you can do after
Evaluate its size-accuracy tradeoff against INT8 and verify tensor dtypes with get_tensor_details().
How it connects
Connects TFLite conversion to INT8 quantization and calibration-data quality.

This explainer was generated by Manning's knowledge graph from the text of Quantization and Fast Inference. The section links point to the parts of the book it draws on.