A PyTorch module’s forward method defines its computation and controls execution order. In DistilGPT2, the model-level method passes embeddings through Transformer blocks and applies final normalization. section 3.1 section 3.1.1
Execution proceeds through nested forward calls:
- GPT2Model iterates through its Transformer blocks. section 3.1.1
- Each GPT2Block calls its attention module and then its MLP module. section 3.1.1
- The block-level methods determine internal operations not shown by the module structure or initialization order. section 3.1.1
Figure 3.2 Anatomy of DistilGPT2. On the left, the general architecture is shown: data flows from the Embeddings layer, passes through the stack of six Transformer Blocks, and ends at the output layer. On the right, a "zoom" into one of those blocks reveals a LayerNorm → GPT2Attention → LayerNorm → GPT2MLP flow, where each module processes the information before passing it to the next.
Use the architecture diagram to trace the model’s data flow and zoom into a Transformer block’s module sequence. section 3.1.1


