liveBook · Manning

Full integer quantization

As explained in Quantization and Fast Inference

Full integer quantization represents model weights and activations using integer formats.

Full integer quantization converts both model weights and activations to INT8. The TFLite converter estimates activation ranges from representative inputs, which you provide as a Python generator; the converter runs calibration internally. section 6.3

Listing 6.5 Representative dataset generator for calibration
def make_representative_dataset(images, num_calibration=200):
    def generator():                                                    #A
        indices = np.random.RandomState(42).choice(
            len(images), size=min(num_calibration, len(images)),
            replace=False
        )
        for i in indices:
            yield [images[i:i+1].astype(np.float32)]                   #B
    return generator                                                    #C

Use a representative dataset generator to supply sample inputs for calibration. section 6.3

On MobileNetV2, full integer quantization produced 3.48× compression and a 1.58 percentage-point accuracy loss. It was slightly larger than dynamic range quantization (3.68×), partly because activation quantization parameters add metadata. For CNN mobile or edge deployment, the decision guide recommends full integer for static INT8 on NNAPI, Coral, or XNNPACK; use strict integer I/O for integer-only hardware. section 6.3

How Quantization and Fast Inference covers it

How deep it goes
Explains representative-data calibration and measured tradeoffs, but does not detail quantization calculations.
Examples
Uses a MobileNetV2 representative-dataset generator and reports compression and accuracy on ImageNette.
What it assumes
Requires representative sample inputs for calibration, supplied through a Python generator.
What you can do after
Configure TFLite conversion with a representative dataset for static INT8 weights and activations.
How it connects
Connects TFLite quantization with calibration, dynamic range quantization, and INT8 deployment on NNAPI, Coral, and XNNPACK.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Quantization and Fast Inference. The section links point to the parts of the book it draws on.