Evidence · DiggingBeagle record
MMA-SafetyBench: multimodal agent safety evaluation
Primary MMA-SafetyBench study of visual semantic hijacking across 1,083 constructed trajectories, nine evaluated systems and five domains. Distinguishes exact attack success from broader semantic compromise and does not independently establish external tool effects.
- Published
- Oct 14, 2026
- Source role
- primary disclosure
Evidence record
Primary MMA-SafetyBench study of visual semantic hijacking across 1,083 constructed trajectories, nine evaluated systems and five domains. Distinguishes exact attack success from broader semantic compromise and does not independently establish external tool effects.
Claim-level citations (3)
- supportsVisual prompt injection redirects simulated multimodal agent decisions: MMA-SafetyBench evaluates 1,083 adversarial trajectories across five domains and nine systems. Its highest reported configuration-specific attack success is 87.96%, not an overall average.
Experimental setup and attack-success results, including the GLM-4.6V video configuration
- supportsVisual prompt injection redirects simulated multimodal agent decisions: The benchmark judges predicted clicks and generated content rather than verifying completed effects in an external environment. Its broader semantic-compromise metric also includes redirection and disruption.
Section 4.2, evaluation metrics and outcome verification
- supportsVisual prompt injection redirects simulated multimodal agent decisions: The study uses separate attack-success and semantic-compromise scores, and reports that adding a safety-priority prompt reduces attack success without eliminating visual semantic hijacking.
Paper abstract, 1-3-5 outcome categories and safety-priority prompt controls