ResNet v2 refined the original ResNet design rather than replacing it. Its pre-activation strategy moves batch normalization and ReLU before convolutional layers, leaving the skip connection unchanged. section 3.0 section 3.4.1
Unlike post-activation blocks, which apply ReLU after the residual and skip paths combine, pre-activation blocks apply activations within the residual branch. The full strategy places batch normalization and ReLU before each convolution and omits activation after addition. section 3.4.1 section 3.4.2
Figure 3.5 Left: the original residual unit shows two “weight” layers (that is, convolutional layers), with batch normalization (BN) and ReLU activations between them, followed by an additional step that merges with the identity connection. Right: the revised residual unit reorganizes the operations by placing BN and ReLU before the weight layers, simplifying the pathway. The gray arrows highlight the direct paths (that is, skip connections) that support the smooth forward and backward flow of information through the residual connections. Used with permission granted by the lead author (Kaiming He).
Compare the original residual unit with the revised ordering; the diagram highlights the direct skip path. section 3.4.1
Ablations found that moving batch normalization after addition disrupted the skip path, while using ReLU alone before convolutions did not meaningfully improve performance. The full strategy supported a 1,001-layer ResNet that reached 4.62% classification error on CIFAR-10. section 3.4.2
