liveBook · Manning

Neural network layer

As explained in Rearchitecting LLMs

A model layer is a computational stage in a neural network that transforms incoming representations.

A neural network layer is a computational stage that transforms a representation. In Transformer models, terminology can vary: pruning workflows often call each indexed Attention-plus-MLP unit a layer, while another convention calls it a block and reserves “layer” for an individual operation such as Linear or LayerNorm. section 2.3 section 4.1

For depth pruning, the practical unit is the complete repeated Transformer block stored in a model’s layer list. Removing blocks shortens the model; choosing which blocks to remove is a separate problem from changing the model’s depth. section 2.3 section 4.1

LAYERS_TO_REMOVE = 2
original_layers_count = len(pruned_model.model.layers)
new_layers_count = original_layers_count - LAYERS_TO_REMOVE
new_layers = pruned_model.model.layers[:new_layers_count]

This snippet calculates the retained layer count and selects the first blocks. The model configuration must also be updated to match the shortened list. section 2.3

Pruning and its trade-off

Removing the final two blocks from an 18-layer Gemma model leaves 16. The resulting model is reported as 4.16% smaller and 10% faster, with benchmark scores falling by 8–12 percentage points. section 2.3 section 2.4

How Rearchitecting LLMs covers it

How deep it goes
Implements block removal and benchmark evaluation, while deferring optimal block selection to importance analysis.
Examples
Compares pruning Gemma from 18 to 16 blocks with a six-block DistilGPT2 architecture.
What you can do after
Apply depth pruning by copying a model, selecting blocks, updating its layer list and configuration, and benchmarking results.
How it connects
Connects layers with Transformer blocks, depth pruning, knowledge recovery, and attention pruning.

Also explained in

Compare how 12 books explain neural network layer →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Rearchitecting LLMs. The section links point to the parts of the book it draws on.