Evidence · DiggingBeagle record
Regressions in self-evolving agent harnesses
Autonomous agent-harness evolution optimized selected benchmark tasks and created regressions on previously solved tasks, indicating that score gains did not imply monotonic capability or safety. Across 16 evolved agents, the study observed each gained between one and 15 solved tasks while losing between two and ten older tasks. The measured performance is benchmark- and budget-dependent. Research/disclosure claims are specific to the documented settings; do not infer unreported victims or additional compromise.
- Source role
- primary disclosure
Evidence record
Autonomous agent-harness evolution optimized selected benchmark tasks and created regressions on previously solved tasks, indicating that score gains did not imply monotonic capability or safety. Across 16 evolved agents, the study observed each gained between one and 15 solved tasks while losing between two and ten older tasks. The measured performance is benchmark- and budget-dependent. Research/disclosure claims are specific to the documented settings; do not infer unreported victims or additional compromise.
Claim-level citations (3)
- supportsEvery tested self-evolving agent gained tasks while regressing on solved tasks: Across 16 evolved agents, the study observed each gained between one and 15 solved tasks while losing between two and ten older tasks. The measured performance is benchmark- and budget-dependent.
Agent evolution experiments; gained and lost tasks; held-out transfer
- supportsEvery tested self-evolving agent gained tasks while regressing on solved tasks: Autonomous agent-harness evolution optimized selected benchmark tasks and created regressions on previously solved tasks, indicating that score gains did not imply monotonic capability or safety.
Agent evolution experiments; gained and lost tasks; held-out transfer
- supportsEvery tested self-evolving agent gained tasks while regressing on solved tasks: The self-evolving-agent study reports that all 16 evaluated evolved agents both gained and lost solved tasks; seven of eight general-task transfer comparisons also lost coverage. The finding concerns selected benchmarks, not a universal inability to improve.
Agent harness evolution regression and held-out transfer evaluation