A PyTorch pre-hook runs before its module’s forward computation and receives the module’s input arguments. Use it to capture a tensor at a specific point in a model; unlike a forward hook, it captures the input rather than the output. section 8.3.1 section 10.1
A typical capture workflow:
- Register a callback on each target module, often using a factory to associate the callback with a layer index. section 8.3.1 section 10.1
- Read the input tensor from args[0] or inputs[0], then detach it before storing it. section 8.3.1 section 10.1
- Keep the returned handles and remove them when capture is finished; otherwise, hooks continue running on later model executions. section 10.1
For attention-layer scoring, pre-hooks on input_layernorm and post_attention_layernorm capture hidden states before those modules execute. The latter input already includes the attention output added through the residual connection. section 8.3.1
Listing 10.2 Hook registration at capture points
def register_hooks(model, target_layers=None):
activations = {}
model._activations = activations
handles = []
def hook_fn(name):
def hook(module, input, output):
tensor = output[0] if isinstance(output,
tuple) else output #A
activations[name] = tensor.detach().cpu()
return hook
def pre_hook_fn(name):
def hook(module, inputs):
activations[name] = inputs[0].detach().cpu() #B
return hook
def capture(key): #C
return target_layers is not None and key in target_layers
for i, layer in enumerate(model.model.layers):
if capture("act_fn") and hasattr(layer.mlp, "act_fn"):
handles.append(
layer.mlp.act_fn.register_forward_hook(
hook_fn(f"act_fn_layer_{i}"))
) #1
if capture("gate_proj") and hasattr(layer.mlp, "gate_proj"):
handles.append(
layer.mlp.gate_proj.register_forward_hook(
hook_fn(f"gate_proj_layer_{i}"))
) #1
if capture("down_proj_input") and hasattr(layer.mlp, "down_proj"):
handles.append(
layer.mlp.down_proj.register_forward_pre_hook(
pre_hook_fn(f"down_proj_input_layer_{i}"))
) #1
....
return handles
See the registration pattern for choosing capture points and saving pre-hook inputs. section 10.1
