Analysis · DiggingBeagle record

The source code can be harmless while the coding agent creates the vulnerability

NameTrap and CyberDual study two different attacks on AI-supported software development: guiding an agent through identifier choices and disguising malicious changes as normal maintenance. Both demand executable security validation beyond fluent review.

Published
Oct 28, 2026

Overview

Code review usually starts by looking for malicious instructions or vulnerable statements that are already present in a change. Agentic development creates another path: a malicious contributor can place seemingly harmless cues in the context from which the assistant later writes insecure code. A reviewer can then miss the defect if the surrounding maintenance story looks routine. Two October papers isolate those stages in controlled experiments.

The dangerous code can appear in the answer, not the input

NameTrap renames identifiers in otherwise nonvulnerable source material and observes how coding agents complete later work. The paper reports attack success ranging from 22.5% to 56.0% across six agent configurations in selected weakness classes, including command injection, cross-site scripting, cryptography and deserialization. The initial change does not have to include the eventual exploit-bearing statement. Identifier experiment.

A scanner that approves the starting patch can therefore be correct about that patch and wrong about the security of the agent's subsequent completion. This is a causal difference from hiding an already malicious code fragment behind formatting or obfuscation.

Realistic maintenance gives a malicious patch cover

CyberDual uses attack-bearing modifications embedded in plausible maintenance work and tests whether AI reviewers find the actual exploitable path. Its researchers constructed 1,088 valid controlled attacks. Individual reviewers detected roughly 85% to 91% under the selected setups, leaving meaningful failure rates even when the review narrative sounded competent. The researchers checked exploit behavior after review rather than awarding success for a persuasive explanation alone. Reviewer evidence.

The paper also reports 28 attacks bypassing all three reviewers in the examined configurations. That is an overlap of failures on controlled repository tasks, not a count of publicly compromised packages. A reviewer may identify the right file yet still miss the actual security invariant, or introduce a repair that passes superficial tests while preserving the exploit. Primary study.

Review the artifact that will actually run

The two studies should not be conflated. NameTrap manipulates the generation input; CyberDual tests review of a malicious output candidate. They share a critical acceptance rule: the security decision must be made on the final executable artifact with its real environment and permissions, not on the apparent harmlessness of earlier context or the reviewer agent's prose. For a security-sensitive change, that means a pinned test of the affected invariant, a known-bad reproduction where available, and a rerun after the reviewer's modifications.

No benchmark here proves that all AI-generated code is insecure or that a particular automated reviewer should be discarded. It shows that each handoff, source context to generated patch and generated patch to accepted build, carries a different failure mode. Treating the two as one 'AI code quality' score would hide the reason the exploit survived.

Research behind this