Analysis · DiggingBeagle record
The attack can finish before the prompt filter sees it
Tool updates, archive processing, retrieved cloud-policy examples and long-run context compaction show why agent security must enforce authority beyond model-visible text.
- Published
- Nov 4, 2026
Overview
A coding agent may never see the bytes that already caused an unwanted side effect. Its tool installer can execute an updated dependency before inference; a retrieved example can silently expand a cloud role; a memory summary can preserve a prohibition without the agent honoring it. These are different failure points. None is fixed merely by telling the model to ignore suspicious instructions.
Locate the first irreversible effect
Four controlled research programs place the decisive trust boundary at different points of an agent workflow:
| Boundary | Source of lower-trust influence | Effect to verify | What the research establishes |
|---|---|---|---|
| Tool installation and preprocessing | Updated tool implementation or opaque archive | Files written, commands executed, resources consumed before model inspection | ETUDE and Step-Zero demonstrate separate harness-level routes under controlled setups |
| Retrieval to generated policy | Poisoned documentation or examples | Effective permissions in a deployable policy artifact | IAMRAGBench distinguishes AWS-valid syntax from the permissions the user intended |
| MCP result to tool execution | Tool descriptions and lower-trust return text | Actual tool dispatch, network contact or secret receipt | MCPVenom separates three evidence tiers rather than calling every model response exfiltration |
| Context compaction to later action | Lossy or adversarial summary of session history | Enforcement of the original restriction at the next effect | ARCBench shows increased violations even when a rule often survives in the summary |
These studies are neither one attack chain nor measurements of the same users. Their settings, denominators and success criteria are different. Their common engineering question is whether a lower-trust input gains authority over something that can commit a real effect.
A tool can do damage without persuading the model
ETUDE reports 97.6% malicious-update success while legitimate tasks completed in 95.9% of its selected trials. The tool's implementation, not merely its readable description, was the attack vehicle. A user could receive an apparently finished answer after the unwanted action. The study does not establish an actual compromised public package registry. Tool-update dossier.
Step-Zero tests a different component: archive expansion and other opaque carriers consumed by agent harnesses. Thirty-two of 42 hostile archive harness-model cells exceeded a 4 MiB processing budget; no matched benign twin did. Twenty-six breaches happened before extracted member content reached the model. A bounded extractor stopped all 28 tested budget breaches in one setup, yet an aligned-carrier test bypassed the front-end guard in one of 28 cells when an agent reached the raw extractor directly. That exception gives the control a precise acceptance criterion: all invocation paths must enforce the same resource and filesystem policy. Archive experiment.
Neither a polite refusal nor a safe-looking assistant transcript proves that earlier package or parser effects were blocked. The relevant records are executable digests, process events, resource counters and sandbox decisions captured outside the conversation.
Valid output can still violate the task
In IAMRAGBench, the attacker's text appeared as reference material for an assistant asked to generate AWS IAM policies. Direct included-context evaluation produced 46 attacker-goal authorization expansions in 292 eligible model-endpoint runs; an end-to-end retrieval setup produced seven in 294. These are distinct experimental routes, not a combined rate. The harmful artifact can be well-formed IAM JSON. The missing check is whether its resulting Action, Resource and conditions remain inside the user-authorized privilege envelope. Policy generation study.
MCPVenom concerns what the model does next rather than only what policy text it writes. Across 9,720 designed cells, 2,113 were classified vulnerable among 9,215 valid susceptible-or-resistant outcomes. Of those vulnerable cells, 1,371 had a dispatched attacker tool call, 487 confirmed network contact and 255 independently verified a canary-secret receipt. A dispatch event is not a completed theft. Model confidence and the presence of a tool are not permission to read a file or contact an external host. Benchmark and scoring tiers.
The rule may survive the summary and still fail
ARCBench reports violations of persistent instructions rising from 9.6% with full context to 37.8% after compaction. Yet 70.1% of compacted runs retained the relevant rule entirely according to the study's rule-state judge. Simply checking whether a prohibition remains in generated memory misses this failure. A later action must be evaluated against authoritative policy state that is not generated by the model or rewritten by its summarizer. Compaction mechanism.
An executor-level acceptance test
The four boundaries require different evidence, not a generic prompt-hardening score. For each delegated job, preserve the original user authorization, the versioned code and artifacts loaded by the harness, the provenance of retrieved text, the policy state across compaction, and the exact external effect at dispatch. Then test four negative controls: a changed tool digest under the same name, an oversized archive reachable through an alternate extractor path, a syntactically valid IAM policy that exceeds a separately defined privilege envelope, and a compacted session in which the summary recommends a forbidden action. Every test must fail closed at the layer that can produce the effect.
A separate SurrealDB MCP session collision is a useful limit on this synthesis. It concerned backend identity binding between JWT users, not a model following injected instructions. It would be misleading to count it as another prompt-injection success, but it tests the same overarching requirement: the final actor and permission are established by the executor, not inferred from conversation text.