BypassGym agents evaded endpoint defenses in 71 of 90 controlled tests

BypassGym measures a capability often missed by coding-agent leaderboards: making an executable loader run under an endpoint protection product instead of merely writing plausible attack code. Five agents built loaders against six endpoint suites and three predetermined payload families in isolated Windows virtual machines, using detection feedback to revise candidates.

The paper reports 71 successful runs out of 90, compared with 3 of 18 unmodified controls. Success included payload execution and the prescribed connection and file-transfer behaviors. It was not a malware campaign: agents received curated offensive templates and the study ran on selected product versions and policies. The contrast across products, including much lower Bitdefender results than Defender, is more informative than a single headline average.

AI-agent offensive capability needs to be scored by completed host effects under realistic defenses, not just code generation.

The 71 successful cells include host execution and prescribed post-execution effects, which is materially stronger than grading only generated source code. It is not a deployment prevalence estimate: the toolchain, available signed binaries and EDR configuration were intentionally controlled. A clean comparison would hold supplied offensive templates and retry budgets constant before claiming an intrinsic model advantage.