liveBook · Manning

Batch normalization folding

As explained in Quantization and Fast Inference

Batch normalization folding is a graph transformation that incorporates batch-normalization parameters into an adjacent operation such as a convolution.

Batch normalization folding absorbs normalization into the preceding layer’s weights and biases. For a convolution, scale each output channel’s weights by γ/√(running variance + ε), and set its bias to (bias − running mean) × scale + β; use zero for a missing convolution bias. Fold before inserting fake-quantization modules: otherwise, quantization-aware training calibrates against unfolded weights, while export uses folded weights with a different distribution and dynamic range. section 5.3.1

Listing 5.9 Batch normalization folding for QAT
def fold_bn_into_conv(conv, bn):
    """Fold batch norm into preceding Conv2d."""
    w = conv.weight.data
    b = conv.bias.data if conv.bias is not None else torch.zeros(
        w.shape[0], device=w.device)
    factor = bn.weight / torch.sqrt(bn.running_var + bn.eps)  #A
    new_weight = w * factor.view(-1, 1, 1, 1)
    new_bias = (b - bn.running_mean) * factor + bn.bias
    new_conv = nn.Conv2d(conv.in_channels, conv.out_channels,
                         conv.kernel_size, conv.stride,
                         conv.padding, bias=True)
    new_conv.weight.data = new_weight
    new_conv.bias.data = new_bias
    return new_conv

Inspect the PyTorch Conv2d folding function, including its handling of missing bias and construction of the folded layer. section 5.3.1

How Quantization and Fast Inference covers it

How deep it goes
Derives per-output-channel Conv2d weight and bias updates and specifies when folding must occur in the QAT workflow.
Examples
Uses a PyTorch function that folds batch normalization into a preceding Conv2d layer.
What you can do after
Apply folding before inserting fake-quantization modules to align trained quantization parameters with folded weights.
How it connects
Connects batch normalization, convolution, quantization-aware training, and fake quantization.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Quantization and Fast Inference. The section links point to the parts of the book it draws on.