A search can lose a useful candidate simply because it scores poorly today. An archive stores candidates and their measured signals so the search can rank past work, recover an earlier branch, or inspect how a result was reached. In an agent improvement loop, the stored candidate might be a version of the system prompt. section 2.1.4 section 4.3.1
Two operations are easy to conflate:
- Record and rank: add scored prompt variations during search; call best() to identify the top artifact. A SQLite-backed archive can persist them, beginning with a genesis prompt. section 2.1.4
- Promote: explicitly call promote() before making a selected artifact available to the agent. Finding a better candidate does not itself deploy it. section 2.1.4
- Revisit: in a Darwin Gödel machine search, keep every variant and sometimes sample a parent from the whole archive rather than only the leaders. Preserve each child's score and parent pointer. section 4.3.5
Why keep a losing branch?
On a toy landscape, a greedy climber stops at a local peak of height 6 because its next step is downhill. Archive search retains the lower valley points as possible parents and reaches a peak of height 10. In a prompt-search run, the same pattern lets a search revisit a low-scoring branch and find wording that excludes both cancelled and returned orders. section 4.3.4 section 4.3.5
Listing 4.8 Never discard, and sample by exploit-or-explore (06_revenue_dgm_loop.py)
def sample_parent(archive): #A
ranked = sorted(archive, key=lambda e: e["score"],
reverse=True)
if random.random() < EXPLOIT_P: # usually exploit
return random.choice(ranked[:3]) # a top candidate
return random.choice(archive) # else explore any member
for it in range(iters):
parent = sample_parent(archive) #B
diagnosis = await reflect(parent["content"], parent["traj"])
child = await mutate(parent["content"], diagnosis)
score, _, traj = await measure_full(child)
child_art = parent["art"].mutate(new_content=child, #C
created_by="dgm")
await store.record( #D
Variant(artifact=child_art, parent=parent["art"],
search_method="dgm"),
GapMeasurement(score=score),
)
archive.append({"art": child_art, "content": child,
"score": score, "traj": traj}) #EThe loop shows the key implementation choice: usually select a leading parent, sometimes select any archived parent, then score and record its child without truncating the archive. section 4.3.5
Retention has a cost. An ever-growing archive means more iterations and model calls when the search repeatedly explores old branches. Spend that budget when a reliable score makes recovery across a deceptive landscape valuable; for an easier or noisier task, a smaller population can be the wiser choice. SQLite is convenient for the initial persistent archive, while a longer-lived production deployment may warrant a database such as Postgres. section 4.3.5 section 2.1.4
