liveBook · Manning

Mean absolute error

As explained in Quantization and Fast Inference

The average absolute element-wise difference between two numerical outputs.

Mean absolute error is the average of the absolute element-wise differences between two numerical outputs. It measures typical divergence when comparing raw model logits. section 6.5

Comparing quantized logits

For ResNet-18, TorchAO INT8 weight-only has a mean absolute difference of 0.021 from the PyTorch FP32 baseline; ORT Static INT8 has 0.135. section 6.5

Listing 6.8 Tensor-level equivalence metrics
def compute_equivalence_metrics(baseline, variant, labels=None):
    diff = baseline - variant                                     #A
    abs_diff = np.abs(diff)
 
    # Per-sample cosine similarity, then average
    dot = np.sum(baseline * variant, axis=1)
    norm_b = np.linalg.norm(baseline, axis=1)
    norm_v = np.linalg.norm(variant, axis=1)
    cos_sim = dot / (norm_b * norm_v + 1e-12)                    #B
 
    preds_base = np.argmax(baseline, axis=1)
    preds_var = np.argmax(variant, axis=1)
    agreement = np.mean(preds_base == preds_var)                  #C
 
    return {
        "max_abs_diff": float(abs_diff.max()),
        "mean_abs_diff": float(abs_diff.mean()),
        "cosine_sim": float(np.mean(cos_sim)),
        "top1_agreement": float(agreement),
    }

The metric is computed by taking the mean of the absolute element-wise differences between baseline and variant outputs. section 6.5

How Quantization and Fast Inference covers it

How deep it goes
Defines the metric and implements it as the mean of absolute element-wise differences, without deriving further statistical properties.
Examples
Reports ResNet-18 mean absolute differences of 0.021 for TorchAO INT8 weight-only and 0.135 for ORT Static INT8.
What you can do after
Evaluate typical logit divergence between a PyTorch FP32 baseline and exported or quantized variants.
How it connects
Contrasts with max absolute difference, mean squared error, cosine similarity, and top-1 prediction agreement.

Also explained in

Compare how 15 books explain mean absolute error →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Quantization and Fast Inference. The section links point to the parts of the book it draws on.