liveBook · Manning

Pre-hook

As explained in Rearchitecting LLMs

A pre-hook is a callback that receives a module's input arguments before the module executes.

A pre-hook is a callback that receives a module’s input arguments before the module executes. In PyTorch, register_forward_pre_hook captures the input; register_forward_hook instead captures the output after execution. section 8.3.1 section 10.1

To capture activations during a forward pass:

  • Choose the module whose input marks the desired capture point, and register a pre-hook that stores inputs[0]. Detach the tensor; move it to the CPU if needed. section 10.1
  • Keep the returned handles, run the model, and remove the hooks afterward. A finally block ensures cleanup even if the forward pass fails. section 10.1
Listing 10.2 Hook registration at capture points
def register_hooks(model, target_layers=None):
    activations = {}
    model._activations = activations
    handles = []
 
    def hook_fn(name):
        def hook(module, input, output):
            tensor = output[0] if isinstance(output, 
                                tuple) else output #A
            activations[name] = tensor.detach().cpu()
        return hook
 
    def pre_hook_fn(name):
        def hook(module, inputs):
            activations[name] = inputs[0].detach().cpu() #B
        return hook
 
    def capture(key): #C
        return target_layers is not None and key in target_layers
 
    for i, layer in enumerate(model.model.layers):
 
        if capture("act_fn") and hasattr(layer.mlp, "act_fn"): 
                handles.append(
                   layer.mlp.act_fn.register_forward_hook(
                       hook_fn(f"act_fn_layer_{i}"))
                ) #1
        if capture("gate_proj") and hasattr(layer.mlp, "gate_proj"): 
                handles.append(
                   layer.mlp.gate_proj.register_forward_hook(
                       hook_fn(f"gate_proj_layer_{i}"))
                ) #1
        if capture("down_proj_input") and hasattr(layer.mlp, "down_proj"):
                handles.append(
                   layer.mlp.down_proj.register_forward_pre_hook(
                       pre_hook_fn(f"down_proj_input_layer_{i}"))
                ) #1
 
    ....
 
    return handles

The registration pattern contrasts output hooks with a pre-hook that stores the first input tensor. section 10.1

Scoring attention layers

Register pre-hooks on input_layernorm and post_attention_layernorm in each decoder block. The first captures the hidden state before attention; the second captures it after the attention output and residual have been added. Compare them with cosine similarity, excluding padding tokens with the attention mask. The importance score is one minus the mean similarity across valid tokens. section 8.3.1

For MLP exploration, a pre-hook on down_proj captures its input, while forward hooks can capture outputs such as gate_proj and act_fn. Check module paths against the model architecture: attention module names can differ across model families. section 10.1 section 10.6

How Rearchitecting LLMs covers it

How deep it goes
Walks through callback registration, tensor storage, forward execution, and handle cleanup, contrasting pre-hooks with forward hooks.
Examples
Uses decoder normalization inputs for attention importance and the down-projection input for MLP activation capture.
What you can do after
Apply it via PyTorch to capture selected module inputs and remove the hooks after a forward pass.
How it connects
Connects activation capture to transformer attention, residual connections, cosine similarity, and MLP projections.

This explainer was generated by Manning's knowledge graph from the text of Rearchitecting LLMs. The section links point to the parts of the book it draws on.