Prompt-injection probes scored benign twins as attacks after learning metadata cues

An injection detector that performs well on held-out rows may still be distinguishing datasets rather than attacker instructions. The shortcut-sensitivity study tested whether properties such as command length, tool naming and passage position predicted benchmark labels. In some setups, inexpensive lexical or metadata features performed unexpectedly well despite lacking the information needed to judge whether a requested action was authorized.

Counterfactual benign twins expose this failure. If a malicious passage is made harmless while keeping its format and metadata nearly unchanged, a detector that still blocks it is likely responding to a noncausal cue. The experiment therefore tests transfer across meaning-preserving and harm-changing transformations rather than rewarding one headline F1 score.

The evidence establishes weakness in the sampled guard evaluations, not a deployed jailbreak or a completed unauthorized state change. A stronger test holds legitimate task context and tool effects constant while changing only the true instruction boundary. Both false denials and missed attacks matter because a guard that blocks nearly everything may look effective on an attack-heavy dataset while being unusable for real work.