A repetition loop is a generation failure in which autoregressive output repeats instead of progressing. Aggressive attention-layer pruning can trigger it immediately. section 8.5.1 section 8.6
Pruning-induced collapse
Removing 15 of 28 attention layers from Llama-3.2-3B yielded about a 6% throughput gain but caused a repetition loop from the first generated token. section 8.5.1
Recovery experiments use knowledge distillation with OptiPFair, varying settings such as alpha/beta balance, calibration sample count, or training duration. Check generated text directly with the Paris prompt; benchmark improvement alone may not establish that generation has recovered. section 8.6 section 8.6
