Evidence · DiggingBeagle record
When Safety Becomes an Attack Surface: Manipulating Abstention
Study of injected false safety/compliance warnings that cause tool-using agents to refuse legitimate tasks.
- Published
- Nov 11, 2026
- Source role
- primary disclosure
Evidence record
Study of injected false safety/compliance warnings that cause tool-using agents to refuse legitimate tasks.
Claim-level citations (2)
- supportsA fabricated safety warning could stop a legitimate agent task without requiring the attacker to request an illicit action: The study inserted compliance or safety warnings into material an agent encountered while completing benign tasks. Instead of causing unauthorized writes, the attacker sought an unnecessary refusal or abandonment; the apparent security justification was carried by a source that was not authorized to define policy. In 51 paired tasks, some models became much more likely to abstain after the warning, while a source-provenance verification step recovered task completion on a small tested subset. One confidentiality control exposed a tradeoff, so indiscriminately challenging warnings can also be unsafe.
AgentAbstain paper, paired safety warnings, 51 tasks and provenance-aware verification results
- supportsA fabricated safety warning could stop a legitimate agent task without requiring the attacker to request an illicit action: These are controlled experimental outcomes, not a rate for real enterprise agents. A genuine safety restriction must still be enforced; the weakness is accepting untrusted text as the origin of that restriction. The evidence is bounded by the cited source and test setting. Broader incidence, other environments, and actual production consequences are not established merely by these observations.
Primary disclosure: results, scope and limitations