chapter three

3 The Gödel Machine, and the importance of signal

 

This chapter covers

  • Measuring the gap between what an agent produced and "good," and why we measure rather than prove
  • Introducing the revenue task and we establish a ground-truth measurement for what good looks like
  • Understanding the significance of the signal family, and multiple signals can be combined
  • Applying search candidate methods to the improved ground truth and reflective signals we have derived

In the early 2000s, Jürgen Schmidhuber described a machine that could improve itself with a guarantee. His Gödel machine would rewrite any part of its own code the moment it could prove the rewrite was an improvement, so it never had to test a change, only prove it. The design was elegant and, aside from toy problems, unusable, because an agent can almost never prove anything about itself in advance. However, it is an anchor that will set the stage for establishing the importance of measuring improvement.

This chapter is about taking our understanding of proof and shifting to more practical applications of measurement. You see, when you cannot definitely prove that a change helps, you certainly can measure whether it did; and when one measured step is not enough, you search for a better version by trying many. That is the whole foundation: measure rather than prove and search rather than solve, and it ends in a method that revives Schmidhuber's dream by relaxing its one impossible demand.

3.1 Measuring the gap

3.1.1 The signal step of self-improvement

3.1.2 The Gödel machine and the proof gate

3.1.3 Why we measure instead of prove

3.1.4 The GapMeasurement union: one shape for every signal

3.2 The agent revenue task

3.2.1 One vague question, six hidden rules

3.2.2 Ground truth: precise, and only as good as the checker

3.2.3 The proof versus ground truth

3.3 The family of signals

3.3.1 Reflection: a critique with no score

3.3.2 The signal ceiling

3.4 Applying signals to improvement

3.4.1 Consuming signals with search

3.4.2 Hill-climbing: the empirical Gödel ratchet

3.4.3 SPO: climbing without a ground truth

3.4.4 Two climbers, one wall

3.5 Exercises

3.6 Summary