Inverse scaling occurs when performance on some benchmarks worsens as models grow, at least temporarily. Task curves can zigzag, plateau, decline, and later climb, rather than improving steadily with scale. section 6.2.1 section 6.2
One possible cause is that larger models latch onto spurious patterns more confidently. Further scaling, better data, or refined training may bring recovery. This contrasts with training loss, which scaling laws predict more reliably than downstream task performance. section 6.2.1 section 6.2
