liveBook · Manning

Specialization

As explained in Rearchitecting LLMs

Domain specialization is the adaptation of a general model to perform effectively on a particular domain or task.

Domain specialization adapts a general model to a particular domain or task. It can reduce the cost and lack of differentiation associated with oversized generic models, but pruning reduces model capacity and can worsen general performance. section 1.5 section 5.2.5

For data-driven width pruning, collect activation norms from domain-specific calibration data. Combine these dynamic activity scores with static weight-magnitude scores to rank neuron pairs, prune the selected pairs, and replace the MLP layers. Check that activation data exists for every layer and update the model configuration to avoid library compatibility issues. section 5.2.4

Listing 5.7 Update model for data-driven pruning
def update_model(model, prune_percent, activation_norms):
    new_intermediate_size = None
    pruning_stats = []
 
    for idx, layer in enumerate(model.model.layers): #A
        mlp = layer.mlp
 
        if idx not in activation_norms: #1
            raise KeyError(f"No activation norms for layer {idx}")
 
        X_d_norm = activation_norms[idx] #2
        original_size = mlp.gate_proj.out_features
 
        new_gate_proj, new_up_proj, new_down_proj, new_size = prune_neuron_pairs(
            mlp=mlp,
            prune_percent=prune_percent,
            X_d_norm=X_d_norm, #3
            layer_idx=idx
        )
 
        mlp.gate_proj = new_gate_proj
        mlp.up_proj = new_up_proj
        mlp.down_proj = new_down_proj
 
        if new_intermediate_size is None:
            new_intermediate_size = new_size
 
    model.config.intermediate_size = new_intermediate_size #4
 
    return model

Shows the layer-by-layer update function, including its activation-data check and the pruning call. section 5.2.4

Wiki and SMS variants

Calibration on wiki text and SMS data produces different pruned Llama-3.2-1B variants, despite the same pruning percentage and parameter count. Each variant has lower perplexity on its own calibration domain than the other specialized variant, but both score worse than the base model. Treat pruning as damage minimization before recovery, not as the complete pipeline. section 5.2.4 section 5.2.5

Table 5.1 Perplexity results
Model
Parameters
Wiki
Perplexity
SMS
Perplexity
Base
1.24B
25.69
122.91
Wiki model
1.07B
(~87%)
36.28
182.54
SMS model
1.07B
(~87%)
48.65
165.54
Static 20% pruned
1.07B
(~87%)
52
217

Compares base, domain-pruned, and static-pruned models by parameter count and wiki and SMS perplexity. section 5.2.5

How Rearchitecting LLMs covers it

How deep it goes
Explains activation-guided MLP pruning and its evaluation, while deferring fine-tuning or distillation as recovery steps.
Examples
Compares wiki-calibrated and SMS-calibrated Llama-3.2-1B variants against base and static-pruned models.
What it assumes
Requires a base model and domain-specific calibration data from which to collect per-layer activation norms.
What you can do after
Evaluate domain-specific pruning by comparing perplexity across calibration domains and against the base model.
How it connects
Connects model pruning with fine-tuning, LoRA, knowledge distillation, and perplexity evaluation.

Also explained in

Compare how 4 books explain specialization →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Rearchitecting LLMs. The section links point to the parts of the book it draws on.