Cosine similarity measures directional alignment between two vectors, independently of their magnitude. It is the dot product of the vectors after normalization. A value of 1 means the vectors point in the same direction, 0 means they are perpendicular, and -1 means they point in opposite directions. section 4.3.2
For depth pruning, compare each transformer block’s input and output activations. A similarity near 1 indicates little directional change; a lower value indicates a larger directional transformation. This is a proxy for block contribution, not a direct measurement of the effect of removing the block. section 4.3.2 section 4.4
Comparing sentence embeddings
Encode three sentences with all-MiniLM-L6-v2, normalize their embeddings, and multiply the normalized matrix by its transpose. In the example, two similarly worded sentences about a cat score 0.6106; a cat sentence and a bus sentence score 0.0563. section 4.3.2
embeddings_normalized = F.normalize(embeddings, p=2, dim=1) #A
# Compute similarity matrix (cosine = dot product of normalized vectors)
similarity_matrix = torch.mm(embeddings_normalized,
embeddings_normalized.T) #B
print(f"Sentence 1 vs 2: {similarity_matrix[0][1]:.4f}")
print(f"Sentence 1 vs 3: {similarity_matrix[0][2]:.4f}")
The PyTorch listing normalizes sentence embeddings and computes their pairwise similarity matrix. section 4.3.2
To rank blocks for task-specific pruning:
- Register forward hooks on each block and run representative calibration data through the model, capturing input and output activations. section 4.3.3
- Compute cosine similarity along the activation dimension, exclude padding with the attention mask, and aggregate valid-token results into block scores. Check for empty inputs, all-padding batches, and non-finite values. section 4.3.3
- Rank low-contribution blocks as removal candidates and compare the resulting model against static selections on the same benchmarks. Protecting boundary blocks and avoiding adjacent removals are selection options. section 4.0 section 4.3.4
The related Block Influence metric is 1 minus the cosine similarity between a block’s input and output. It needs forward inference rather than the gradient calculations used by a Taylor approximation. Rankings depend on calibration data: WinoGrande and WikiText selected different fourth blocks for removal from Qwen3-0.6B. section 4.4 section 4.3.4
























