Layer normalization recenters and rescales activations at each layer to stabilize their scale. In a Transformer encoder, each sub-layer’s output is written as LayerNorm(x + SubLayer(x)), combining normalization with a residual connection. section 9.6
Figure 9.8 Transformer architecture.
The architecture figure shows where layer normalization sits in the Transformer encoder. section 9.6











