Analysis · DiggingBeagle record

The dangerous instruction may no longer be visible

Repository poisoning, malicious model adaptation and reused reasoning caches place attacker influence beyond the visible prompt, but each requires a different trust-boundary test.

Published
Oct 20, 2026

Overview

A coding agent can encounter a malicious instruction without seeing a suspicious message in the current chat. The attack may live in a repository file that the agent executes, in the selected model's weights, or in inference state retained from an earlier interaction. These locations have different owners and require different evidence of compromise.

Four October research demonstrations make the distinction testable. They are controlled experiments, not evidence that the named systems were compromised in production.

A repository can turn ordinary maintenance into execution

TrojanForge selects code locations likely to be reached by a coding agent's legitimate workflow. A malicious component need not persuade the model to repeat a command: it can be executed while the agent builds, tests or imports the project. The researchers evaluated 384 poisoned repositories across several languages and harnesses, but those repositories were experimental targets, not 384 reported victim projects.

The security event is a sequence: untrusted repository material is read, an agent chooses a task-related action, and a runtime executes an attacker-influenced path. A scan of the agent's latest prompt cannot establish the provenance of every dependency, test fixture or command that execution will touch.

Model weights can carry a policy that tools make dangerous

In a malicious LoRA study, researchers deliberately adapted Qwen3-8B so that a shell-enabled agent read a secret canary and emitted it through DNS. The resolver log, rather than the model's explanation, was used to check the observed effect. Blocking non-DNS web requests did not close that tested channel; constraining DNS egress altered the outcome. No maliciously distributed public checkpoint was established.

RAGLeak uses a different model-weight backdoor. With access to private retrieved documents and an attacker-facing question channel, the modified generator outputs a reversibly transformed version of retrieved information. The authors report 58% to 100% attack success across selected model and dataset settings, not leakage from real healthcare or corporate victims. Output that looks unlike the original secret may still disclose it to someone who can decode the transformation.

Both experiments place the adversarial behavior in a model artifact. Their privileges are not interchangeable: the first requires shell, secret access and resolvable outbound DNS; the second needs an improperly bounded private retrieval and response path. A safe serialization format alone cannot establish safe behavior once a model starts choosing actions.

A removed instruction may survive through cache lineage

R2ECAP targets reused key-value representations of earlier reasoning. An attacker shapes an earlier trace, which influences later generation even when the malicious text is no longer presented as an explicit prompt. The paper reports success up to 79% in exact-query cache reuse and lower but nonzero rates across changed queries. It does not demonstrate a cross-tenant compromise in a commercial model service.

Deleting visible text and discarding the derived state are not equivalent operations. Cache reuse creates its own provenance problem: who generated the earlier representations, which request may reuse them, and whether a tenant, task or permission boundary was crossed.

Match the boundary to the control

Attacker-influenced material Decisive transition Verification target
Repository code and assets Trusted agent tool executes or modifies untrusted project content Provenance of dependencies and actual process, filesystem and network effects
Adapted model weights Generator chooses a secret-reading or disclosure behavior Artifact identity plus independent tool, retrieval and egress restrictions
Reused reasoning cache Old derived state becomes input to a new generation Cache ancestry, task isolation and invalidation on trust changes

These mechanisms should not be collapsed into one prompt-injection success rate. The first may produce code execution without a convincing message; the second changes generator behavior; the third persists influence after its original text disappears. The common requirement is independent evidence of the effect and an enforceable boundary outside the untrusted material.

Research behind this