register_forward_hook attaches a callback to a PyTorch module. The callback runs after the module’s forward computation and receives the module, its inputs, and its output. section 4.6
Use a forward hook to:
- Inspect module inputs and outputs, such as tensor shapes. section 4.3.1
- Capture activations for analysis or modify data during execution. section 4.3.1 section 10.1
- Keep the returned handle and call its remove() method when capture is no longer needed; hooks otherwise persist on the model. section 4.3.1 section 10.1
Inspecting a decoder block
Register a shape-printing callback on the first decoder layer, then run a tokenized prompt through the model. The example reports input and output shapes of [1, 8, 1024]. section 4.3.1
def print_shape_hook(module, input, output):
input_tensor = input[0] #A
print(f"Module class: {module.__class__.__name__}")
print(f"Module id: {id(module)}")
print(f"Input shape: {input_tensor.shape}")
print(f"Output shape: {output.shape}")
Callback example showing how to inspect the module and its input and output shapes. section 4.3.1
