Safety monitors can detect dangerous input yet fail to enforce their decision

A security guard may remain configured and detect its test payload yet still fail to block the action. The Monitor-Validity paper collected 18 confirmed parent-versus-fixed regression comparisons across 11 LLM or agent framework groups. It separates evidence-path failures that reached enforcement boundaries from missing observability or verifier channels. For example, a positive regex match existed, but an adapter did not forward it to a block decision.

This is neither a generic jailbreak benchmark nor 18 reported exploited production incidents. The researchers selected checkable repair cases rather than a representative sample of all frameworks. Their canary method is useful precisely because it checks each link between the altered schema or stream and the final action, not just whether a detector emits a warning.

A detector match is an intermediate observation, while a denied tool effect is an enforcement result. The paper's paired regression cases were chosen because a fix or parent version exposed a change in that path. Their count is not a prevalence estimate across agent frameworks. Versioned canary checks should therefore exercise each adapter and streaming transition and assert the final blocked action, rather than merely search for a warning in logs.