liveBook · Manning

Representative dataset

As explained in Quantization and Fast Inference

A representative dataset is a collection of sample inputs that reflects the data a model will process and supports quantization calibration.

A representative dataset supplies sample inputs for activation-range calibration during full-integer TFLite quantization. TFLite manages calibration internally; about 200 samples suffice for most vision models, with diminishing returns beyond that. Static calibration is unsuitable for models with high activation variance, such as transformers. section 6.3

Listing 6.5 Representative dataset generator for calibration
def make_representative_dataset(images, num_calibration=200):
    def generator():                                                    #A
        indices = np.random.RandomState(42).choice(
            len(images), size=min(num_calibration, len(images)),
            replace=False
        )
        for i in indices:
            yield [images[i:i+1].astype(np.float32)]                   #B
    return generator                                                    #C

See a MobileNetV2 generator that selects up to 200 images without replacement and yields batch-one float32 inputs. section 6.3

How Quantization and Fast Inference covers it

How deep it goes
Defines a generator-based calibration handoff and recommends about 200 vision samples, but leaves calibration internal to TFLite.
Examples
Uses a MobileNetV2 generator that selects up to 200 images without replacement and yields batch-one float32 inputs.
What you can do after
Apply it by passing a Python generator of representative inputs to the TFLite converter for full-integer quantization.
How it connects
Connects activation-range calibration with full-integer quantization and contrasts TFLite’s internal loop with PyTorch PT2E’s explicit pipeline.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Quantization and Fast Inference. The section links point to the parts of the book it draws on.