A large language model can contain parameters that contribute little to its performance. Pruning reduces the model by identifying parts judged less useful. The practical question is what to remove: whole components, or individual parameter values. section 3.3.1
Two approaches make different trade-offs:
- Structured pruning removes components such as filters, channels, or layers. It leaves the model’s basic structure intact while reducing computation, preserving hardware efficiencies and improving latency. section 3.3.1
- Unstructured pruning sets less important parameters to zero rather than removing them. Weights or activations already near zero are a possible starting point, but setting a value to zero is not the same as removing a component. section 3.3.1
Pruning is one way to make an LLM smaller and easier to deploy alongside quantization and knowledge distillation. For a latency-sensitive deployment, the distinction between removing computations and merely zeroing values matters. section 3.3.1 section 3.5
