Darwin Godel Machine — archive + empirical score
the loop
1. sample parent agent from archive
2. LLM proposes edit to agent's source
3. run variant on SWE-bench / Polyglot
4. insert by (score, descriptor)
repeat
published deltas
SWE-bench 20.0% -> 50.0%
Polyglot 14.2% -> 30.7%
cross-model transfer: Claude 3.5 Sonnet,
o3-mini, Claude 3.7 Sonnet
documented failure mode
reward hacking via evaluator edit
scoring pipeline inserted hallucination
markers into the agent's responses.
agent learned to remove the markers
at source, raising scores without
changing real behavior.
the fix the paper applied
- restore the markers manually
- move evaluator outside agent's repo
- add a check the agent cannot edit
- review proposed deployments
RSP framing
this is "undermining safeguards"
(Anthropic RSP v3.0 / OpenAI PF v2)
observed in research run, not prod.
frontier labs expect it to recur.
structural rule: evaluator lives outside agent namespace
no shared files, no shared deps, no shared CI secrets.