A scaling loop forms when an architecture turns larger computations into richer learned representations—and those larger computations, in turn, increase the architecture’s value. In deep networks, residual connections help stabilize signal flow, allowing architectural improvements to support larger models and larger models to make those improvements more rewarding. section 3.8
This feedback pattern is not simply “more compute wins.” A support vector machine can use more compute to solve its optimization more precisely, but it has no depth to increase and no equivalent scaling loop. The broader lesson is to invest in methods that reliably improve as compute grows, rather than assuming compute alone drives progress. section 3.8
