Analysis · DiggingBeagle record
A prompt-injection defense can win the attack benchmark and lose the user's task
Resource scoping, input sanitation and instruction-channel training reduce different errors. Their security metrics cannot replace authorized-task completion.
- Published
- Oct 31, 2026
Overview
One defense experiment moved successful injected attacks from 93 of 560 to 1 of 560. On the same comparison, legitimate task completion fell from 47 of 60 to 24 of 60. The Realm study makes an uncomfortable but useful tradeoff visible: blocking an agent's resources can remove attacker opportunities while also denying authorized work. That is not a reason to abandon access control. It is a reason to measure both effects on the same task set.
Three different interventions
| Intervention | Boundary being changed | Evidence in the selected research | What the result does not show |
|---|---|---|---|
| Realm resource scoping | Which assets and tools the agent may reach | Attacks fell from 93/560 to 1/560; legitimate completions fell from 47/60 to 24/60 | Production utility under a fully configured asset inventory |
| SEAL input sanitation | What lower-trust text reaches the decision agent | Selected AgentDojo attack success fell from 34.6% to 0.1% | Equivalent protection across untested attacks and tools |
| LLMON typed instruction/data post-training | How the model represents instructions versus task data | Large indirect-injection improvements in selected experiments | Hard runtime authorization or equal performance in direct and multi-turn attacks |
The Realm paper, SEAL paper and LLMON paper do not use one shared benchmark denominator. Their numbers are not a ranking of three competing products. They test different points in a system, and each one can leave a capability reachable through another channel. Realm still reported a successful attack through an allowed calendar tool, which is a reminder that resource eligibility and action intent are different checks.
Training is not a history eraser
A separate conditional-backdoor study planted behavior in a small fraction of pretraining data and found that a trigger could survive later clean supervised and preference-style training. The reported 0.1 percent poison mixture is an experimental setup, not a universal compromise threshold or evidence about a named commercial model. Its relevance here is temporal: a model's current instruction behavior may not reveal how a hidden condition was introduced during earlier training. A test that checks only clean prompts can miss the trigger entirely.
A usable acceptance test
An application deploying any of these controls should report four results on the same held-out environment: authorized tasks completed, unauthorized tool effects realized, attempts rejected before execution, and the human or system intervention needed when a safe refusal leaves legitimate work unfinished. Collect trace-level evidence for the actual tool calls. A lower injection success percentage is valuable only when it is interpreted alongside utility, reachable capability and unseen adversarial variants.
The sanitizer case and typed-channel case show behavioral reductions; the Realm case shows a capability restriction; and the pretraining experiment tests persistence across training stages. Treating those four results as one generic safety score would conceal what each control actually changes.