Every tested self-evolving agent gained tasks while regressing on solved tasks

Autonomous agent-harness evolution optimized selected benchmark tasks and created regressions on previously solved tasks, indicating that score gains did not imply monotonic capability or safety.

Across 16 evolved agents, the study observed each gained between one and 15 solved tasks while losing between two and ten older tasks. The measured performance is benchmark- and budget-dependent.

The study treats task coverage as a changing set, not one scalar score. Across 16 self-evolved agents, every system gained between one and 15 newly solved benchmark tasks but also lost between two and ten tasks solved by its earlier version. Some agents improved repeated-run reliability while losing coverage in general tasks. The result therefore does not prove that harness evolution always makes agents worse; it shows why average success rate can hide regressions on previously reliable actions. Production admission should pin an independent suite of previously passed work, especially irreversible tool and authorization behaviors, instead of assuming the next agent version dominates its predecessor because its headline benchmark improved.

A single aggregate benchmark improvement conceals task-level losses. In the reported set, each of 16 evolved agents gained some solved tasks and lost others, so neither a leaderboard gain nor a best-parent selection ensures monotonic improvement. A deployable harness change needs a protected regression set and a separate held-out transfer test, especially for tool permissions and irreversible effects. This is different from RoboDojo's grader-exposure result: one identifies coverage regressions under evolution, the other identifies a misleading measure of task success.