CyberArena finds agent patches can pass stale checks and miss alternate attack paths

CyberArena places attacker and defender agents in an isolated, changing service environment. A defender must identify and repair weaknesses without breaking legitimate functionality, while an opponent probes alternative routes after patches. This setup is harder than replaying one known proof of concept against a static service: a restarted process or failed old exploit can create a false sense of closure while another reachable vulnerability remains.

The authors evaluated 466 valid matches across 40 challenge origins and 12 agents. Another 294 runs were excluded, so they are not in the scored denominator. The analysis also describes six accepted captures attributed to scoring shortcuts among 1,336 accepted captures. That is an arena-integrity issue, not a claim of six real victim breaches. Detailed trajectory interpretation relied partly on model auditors rather than comprehensive independent human review.

The practical distinction is between attempted patching, service recovery, failed replay of a stale attack and actual closure of all relevant attack paths. The study establishes agent failure modes under contest conditions, not the probability that an enterprise will resist an adaptive attacker. Stronger validation would preserve the postpatch service state, attempt a different exploit route and separately verify that the useful service still functions.