MMA-SafetyBench modifies what a multimodal agent sees during an otherwise legitimate task. Attacker-written instructions appear as plausible dialogs, page elements, document content or video frames, including forged authority cues. The model is supposed to interpret the image as task data; the attack seeks to promote selected visual content into…
Inspect the ClaimsCase · DiggingBeagle record
Visual prompt injection redirects simulated multimodal agent decisions
Scope
Constructed multimodal trajectories over five benchmark domains and nine evaluated systems. Reported ASR and semantic compromise concern benchmark outputs, not verified production data theft, executed transfers or actual remote tool side effects.
Assessment profile
Assessment method- Setting
- evaluation
- Exploitation
- demonstrated
- Evidence
- Grade C
- Remediation
- not applicable
- AI role
- AGAINST AI
Basis and provenance
The primary abstract describes controlled injected visual observations and output-level measurements. Successful compromise is confined to constructed benchmark responses; the full trial corpus, effect oracles and actual tool side effects were not independently inspected. Impact remains ungraded because the available evidence does not establish realized harm or a defensible general severity for the assessed deployment.
Assessed Oct 16, 2026 using diggingbeagle.assessment/1.
Evidence basis: CLM-MMA-VISUAL-INJECTION-01 · CLM-MMA-VISUAL-INJECTION-02 · CLM-MMA-VISUAL-INJECTION-03 · MMA-SafetyBench: multimodal agent safety evaluation
At a glance
An interface screenshot is evidence about a task, not a trusted authority source. Prevention needs a separation between observed visual content and permissible state changes, plus independent controls at tools that can affect files, accounts or networks. A better refusal prompt is not such a control.
Read the implicationsThe study does not quantify attacker-controlled images causing completed file operations, credential exposure or payments on live external services. Model-specific maximum ASR and broader semantic compromise use different criteria and must not be pooled as a single attack prevalence. Changes in the screenshot renderer, OCR path, GUI sandbox and tool…
Limits and uncertaintyFull account
MMA-SafetyBench modifies what a multimodal agent sees during an otherwise legitimate task. Attacker-written instructions appear as plausible dialogs, page elements, document content or video frames, including forged authority cues. The model is supposed to interpret the image as task data; the attack seeks to promote selected visual content into instructions controlling its next action.
The researchers construct 1,083 adversarial trajectories across web, desktop, mobile, document and video workflows, then score the resulting agent responses. The largest reported attack-success rate is 87.96% for a particular model and configuration, while the broader semantic-compromise maximum is 98.09%. These are not pooled averages. The latter metric can count redirection or disruption even without an exact attacker-target match. The study also finds that a safety-priority instruction reduces, but does not remove, this type of output-level vulnerability. Completing an unauthorized external operation would require additional tool permissions and effect verification not supplied by these output judgments.
Mechanism and trust boundary
- 01
Place instructions in a lower-trust visual surface
A forged alert, dialog, page component, frame or document element appears in an image observed during a legitimate task.
Boundary: attacker-controlled scene / observed input
- 02
Model treats a visual cue as an instruction
The agent interprets the content as relevant authority rather than solely as data describing the environment.
Boundary: perception / agent decision
- 03
Output diverges from the authorized goal
The benchmark scores target-matching outputs or broader semantic disruption. It does not independently verify a completed external effect.
Boundary: agent output / external action authority
Timeline
- Oct 14, 2026disclosure
Public disclosure
Claims & evidence
CLM-MMA-VISUAL-INJECTION-01MMA-SafetyBench evaluates 1,083 adversarial trajectories across five domains and nine systems. Its highest reported configuration-specific attack success is 87.96%, not an overall average.supported
Basis: reported finding
- supportsMMA-SafetyBench: multimodal agent safety evaluationprimary disclosure
Experimental setup and attack-success results, including the GLM-4.6V video configuration
CLM-MMA-VISUAL-INJECTION-02The benchmark judges predicted clicks and generated content rather than verifying completed effects in an external environment. Its broader semantic-compromise metric also includes redirection and disruption.supported
Basis: reported finding
- supportsMMA-SafetyBench: multimodal agent safety evaluationprimary disclosure
Section 4.2, evaluation metrics and outcome verification
CLM-MMA-VISUAL-INJECTION-03The study uses separate attack-success and semantic-compromise scores, and reports that adding a safety-priority prompt reduces attack success without eliminating visual semantic hijacking.supported
Basis: reported finding
- supportsMMA-SafetyBench: multimodal agent safety evaluationprimary disclosure
Paper abstract, 1-3-5 outcome categories and safety-priority prompt controls
Implications
An interface screenshot is evidence about a task, not a trusted authority source. Prevention needs a separation between observed visual content and permissible state changes, plus independent controls at tools that can affect files, accounts or networks. A better refusal prompt is not such a control.
Controls and mitigations
- Test visual instructions alongside the legitimate goal under the same observation representation used in production.
- Apply externally specified tool permissions to actions even when the agent claims the screenshot contains a system instruction.
- Score exact unauthorized external effects separately from textual or visual answer redirection.
Unknowns and contradictions
- The study does not quantify attacker-controlled images causing completed file operations, credential exposure or payments on live external services.
- Model-specific maximum ASR and broader semantic compromise use different criteria and must not be pooled as a single attack prevalence.
- Changes in the screenshot renderer, OCR path, GUI sandbox and tool privileges may materially alter realized outcomes.
Sources and citation
Material revision history
- Oct 16, 2026 · Canonical change recorded · new in release · revision 160