A workload is memory-bound when moving data, rather than performing arithmetic, limits performance. During LLM inference, GPUs can spend much of their time waiting for data from memory, leaving arithmetic capacity underused. section 4.1.2
Depth pruning targets this bottleneck by removing entire Transformer blocks. Each removed block eliminates a round of loading data from VRAM, processing it, and writing results back. It also reduces model memory and the KV-cache, which stores attention states for generated tokens. section 4.1.2 section 4.6
Removing layers reduces cache growth
Removing 25% of a model’s layers reduces KV-cache memory growth by 25% for each additional generated token. For Qwen3-06B, each Transformer block accounts for 2.1% of model size. section 4.1.2

















