Self-improvement means developing better capabilities or performance through analysis, learning, or modification. For LLMs, the idea is plausible but sharply constrained: generated outputs reproduce the training distribution imperfectly rather than adding genuinely new information. section 7.1.1
A proposed LLM self-improvement loop looks like this:
- Train an LLM on an initial dataset. section 7.1.1
- Generate additional data with the trained LLM. section 7.1.1
- Add that data to the training set, retrain or fine-tune the model, and repeat. section 7.1.1
- Expect degradation rather than indefinite improvement when each round supplies only noisy, incomplete copies of the original information. section 7.1.1
Why external information matters
A model that repeatedly trains on its own generated material is recycling an imperfect reproduction of its earlier data. Sustained improvement requires external or genuinely new information at each round, along with resources such as data, power, and computational capacity. section 7.1.1
Claims of unlimited self-improvement extend the idea from a model making a better model to an endless chain of increasingly capable models. That argument depends on uncertain assumptions about intelligence, resources, and whether intelligence has a single axis of improvement. Existing evidence instead points toward computational difficulty and diminishing returns. section 9.2 section 9.2.1
Figure 9.2 The S-curve, or sigmoid, shows the classic plateau behavior: at some point, you hit diminishing returns. The counterpoint to this expressed by the iterative S-curve model is that progress continues past the plateau of diminishing returns by discovering new techniques, each represented by a new S-curve. The new techniques may start worse than the existing methods but have a higher potential to surpass them.
Compare ordinary S-curve progress, where improvements plateau, with iterative S-curves, where new techniques begin a fresh period of gains. section 9.2.1
