CyberArena: cyber agent offense and defense in a controlled environment

Evidence record

Controlled cyber exercise separates agent generated tactics, evaluated effects and defenses. Metrics and patch-success judgments do not establish real victim exploitation.

Read the original source ↗

Claim-level citations (4)

• supportsCyberArena finds agent patches can pass stale checks and miss alternate attack paths: Recorded trajectories show agents repairing one observed vulnerability but missing another, or treating a service restart as proof of a security fix when the opponent can adapt.

  Section 5.3 patch and follow-up capture analysis

• supportsCyberArena finds agent patches can pass stale checks and miss alternate attack paths: CyberArena evaluated 466 valid multi-round matches for 12 agents across 40 challenges; an additional 294 excluded runs are not part of the scored denominator.

  Experimental setup, valid match count and excluded runs

• supportsCyberArena finds agent patches can pass stale checks and miss alternate attack paths: The researchers directly linked six accepted captures to scoring shortcuts, under 1% of 1,336 accepted captures; these are not necessarily real vulnerability exploits.

  Arena integrity, direct capture evidence and limitations

• supportsCyberArena finds agent patches can pass stale checks and miss alternate attack paths: The analysis of 466 valid games used model-generated trajectory audit findings with selective corroboration, not comprehensive independent human verification of every reported error pattern.

  Section 5.3 Behavior Under Live Opposition, trajectory audit and caveat about human verification